Massively Parallel Processing

Massively Parallel Processing

Though I haven’t been living under a rock, today was the first I heard about Massively Parallel Processing. I honestly thought it was a joke (a strange-sounding name, to me at least) but apparently it’s very real and is, essentially, a multiprocessing architecture that uses up to thousands of processors.

Our dear friends at WikiPedia have this to say:

Massive parallel processing (MPP) is a term used in computer architecture to refer to a computer system with many independent arithmetic units or entire microprocessors, that run in parallel. The term massive connotes hundreds if not thousands of such units. Early examples of such a system are the Distributed Array Processor, the Goodyear MPP, the Connection Machine, and the Ultracomputer.

Some years ago many of the most powerful supercomputers were MPP systems.

In this class of computing, all of the processing elements are connected together to be one very large computer. This is in contrast to distributed computing where massive numbers of separate computers are used to solve a single problem.

The earliest massively parallel processing systems all used serial computers as individual processing elements, in order to achieve the maximum number of independent units for a given size and cost.

Through advances in Moore’s Law, single-chip implementations of massively parallel processor arrays are becoming cost effective, and finding particular application in high performance embedded systems applications such as video compression. Examples include chips from Ambric, picoChip, and Tilera.

The good people at Answers.com go into a little more detail (with images):

(Massively Parallel Processing or Massively Parallel Processor) A multiprocessing architecture that uses up to thousands of processors. Some might contend that a computer system with 64 or more CPUs is a massively parallel processor. However, the number of CPUs is not as much the issue as the architecture. MPP systems use a different programming paradigm than the more common symmetric multiprocessing (SMP) systems used as servers.

In an MPP system, each CPU contains its own memory and copy of the operating system and application. Each subsystem communicates with the others via a high-speed interconnect. In order to use MPP effectively, an information processing problem must be breakable into pieces that can all be solved simultaneously. In scientific environments, certain simulations and mathematical problems can be split apart and each part processed at the same time. In the business world, a parallel data query (PDQ) divides a large database into pieces. For example, 26 CPUs could be used to perform a sequential search, each one searching one letter of the alphabet.

To take advantage of more CPUs in an MPP system means that the specific problem has to be broken down further into more parallel groups. However, adding CPUs in an SMP system increases performance in a more general manner. Applications that support parallel operations (multithreading) immediately take advantage of SMP, but performance gains are available to all applications, simply because there are more processors. For example, four CPUs can be running four different applications. See SMP.

MPP and SMP Architecture

In MPP operation, the problem is broken up into separate pieces, which are processed simultaneously. In SMP, CPUs are assigned to the next available task or thread that can run concurrently.

An Integrated Architecture

This early Reliant 1000 from Pyramid (later Fujitsu Siemens) was an MPP machine that was combined with SMP systems to provide both kinds of processing. The SMP systems attached to the Reliant’s high-speed mesh interconnect, providing an unusual degree of flexibility and scalability. (Image courtesy of Pyramid Technology Corporation.)

ServerNet Interconnect

This is a conceptual diagram of the advanced ServerNet interconnect architecture from Tandem (later acquired by Compaq), which functioned as a high-speed switch between CPUs and I/O subsystems. It was used for both MPP applications and clusters of SMP systems. The purple and orange units are the processors and disks. The green pipes are the ServerNet mesh. (Image courtesy of Tandem Computers Incorporated.)