Parallel Programming Protocols

Three protocols are presented here for parallel programming, one which has become the standard, one which used to be the standard, and one which some feel might be the next big thing.

MPI

• Reasons to do it
  1. You want to use the most common parallel protocol in current use (as of this writing, July 2002).
  2. You want to be able to use the most efficient parallel protocol in common use.
• Reasons not to do it
  1. You like to buck trends.

For a while there, the parallel protocol war was being waged over PVM and MPI. By most everyone's account, MPI won. It is a highly efficient and easy to learn protocol that has been implemented on a wide variety of platforms. One criticism is that different implementations of MPI don't always talk to one another. However, most cluster install packages give you both of the two most common implementations (MPICH and LAM/MPI). If you are setting up a small cluster, you can choose freely between either, they both work well, and as long as you use the same version of MPI on each machine, you will not have to rewrite any MPI code.

MPI stands for Message Passing Interface. Basically, independent processes send messages to each other.

Both LAM/MPI and MPICH simplify the process of starting large jobs on multiple machines.

PVM

• Reasons to do it
  1. You want a process which can easily recursively spawn other processes.
  2. You want a simple implementation that works across different platforms.
  3. You have legacy code in PVM that you don't want to modify.
• Reasons not to do it
  1. MPI has become the new standard, and is better supported.

PVMs daemon based implementation makes it easy to start large jobs on multiple machines. PVM was the first standard for parallel computing to become widely accepted. As a result, there is a large amount of legacy code in PVM still available.

PVM also allows for the ability to spawn multiple programs from within the original program.

JavaSpaces

• Reasons to do it
  1. You need to pass not just data, but instructions on what to do with that data.
  2. You want an object oriented parallel framework.
  3. Using buzzwords like Linda and TupleSpaces will make you sound cool.
• Reasons not to do it
  1. Your need a high computational efficiency.
  2. You are afraid that the other parallel programmers will tease you.

Java is a versatile computer language that is object oriented and is widely used in computer science schools around the country. JavaSpaces is Java's parallel programming framework which operates by writing entries into a shared space. Programs can access the space, and either add an entry, read an entry without removing it, or take an entry.

Java is an interpreted language, and as such typical programs will not run at the same speed as compiled languages such as C/C++ and Fortran. However, much progress has been made in the area of Java efficiency, and many operating systems have what are known as just-in-time compilers. Current claims are that a well optimized java platform can run java code at about 90% of the speed of similar C/C++ code.

Java has a versatile security policy that is extremely flexible, but also can be difficult to learn.

JavaSpaces suffers from high latency and a lack of network optimization, but for embarrasingly parallel problems that do not require synchronization, the JavaSpaces model of putting jobs into a space, letting any "worker" take jobs out of the space, and having the workers put results into the space when done leads to very natural approaches to load balancing and may be well suited to non-coupled highly distributed computations, such as SETI@Home.

JavaSpaces does not have any simple mechanism for starting large jobs on multiple machines, however, because you can pass instructions along with data, JavaSpaces is well suited for the creation of a generic worker-foreman model, in which generic workers exist permanently on many nodes, and custom "foremen" send out data to analyze and specific instructions on how to do just that.