It is now clear that parallel computing is here to stay: our voracious need for ever greater computing power simply cannot be satisfied by conventional, single-processor architectures. More and more companies are investing in architectures with multiple processors, and more and more colleges and universities are including parallel computing in their curricula.
The increase in the use of parallel computing is being accelerated by the development of standards for programming parallel systems. Developers can now write portable parallel programs, and hence expect to obtain a reasonable return on the huge investment required in a large parallel software development project.
Parallel Programming with MPI or PPMPI is first and foremost a ``hands-on'' introduction to programming parallel systems. It was written for students and professionals who have no prior experience in programming parallel systems. It was designed for use both as a self-paced tutorial and as a text in a more conventional classroom/computer laboratory setting. The only pre-requisite to reading it is a nodding acquaintance with the first-year college math sequence and a knowledge of a high-level, procedural computing language. PPMPI provides both a complete introduction to MPI and an elementary introduction to parallel programming. It covers all the features of MPI, it provides a brief overview of parallel computing, and it provides an introduction to such topics as parallel debugging, parallel program design and development, and parallel program performance analysis. It also contains an introduction to the use of MPI libraries. In the belief that ``teaching by example'' is the most useful approach, all of the concepts are introduced through the use of fully developed program examples. The source for all of the programs can be downloaded from http://www.cs.usfca.edu/mpi.
Except for the material in Chapters 3 - 8 the chapters are mostly self-contained and can be read in any order. Chapters 3 - 8 form a self-contained tutorial introduction to MPI, and, as such, should probably be read in order, and before the remaining chapters. There are, however, parts of these chapters that can be omitted with no loss of continuity. The material on gather, scatter, and allgather in Chapter 5 and the material on topologies in Chapter 7 can be safely omitted on a first reading.
Before actually writing a larger parallel program, you will probably want to at least familiarize yourself with some of the problems involved in carrying out I/O on parallel systems. So Chapter 8 should receive at least a cursory initial examination. If you're anxious to make use of non-blocking communications, Chapter 13 can be read at any point after completing Chapter 7. Section 1.5 provides a more detailed overview of the contents of each chapter.
Since space considerations don't permit the presentation of programming examples in both C and Fortran, a choice had to be made between the two. For students, using C is the clear choice, since most learn to program in Pascal or C++, both of which are closer to C than to Fortran 77. However, Fortran would probably be the language of choice for most practicing scientists and engineers. Believing that the greater experience of the professional audience will make it easier for them than for students to follow examples in an unfamiliar language, I decided to use C throughout. However, I have tried to write all of the C source in a style that should be relatively accessible to a Fortran programmer. I have made very limited use of pointers and dynamic memory allocation, and I have tried to avoid the use of C's more obscure constructs. In addition, all of the example programs are available online in Fortran 77, thanks to a former student, Laura Koonce, who has done the translation. They can be downloaded from http://www.cs.usfca.edu/mpi.
All of the examples in the text have been written in ANSI C rather than Kernighan and Ritchie C. So if your system uses a K&R C compiler, you will probably want to get a different compiler. The GNU C compiler, gcc, uses ANSI C, it is freely available from a number of sites on the internet, and it has been ported to virtually all currently available systems.
All of the programs in the text have been tested on a network of Silicon Graphics workstations running the mpich implementation of MPI. They have also been tested on an nCUBE 2 running a slightly optimized version of mpich. Please report any errors that do surface, in either the code or the text, to peter@usfca.edu. I'll pay the usual bounty of $1 to the first person reporting each error. A list of errata will be available at http://www.cs.usfca.edu/mpi/errata.html.
We also teach an upper division class in parallel programming at USF. PPMPI is followed more or less on a chapter-by-chapter basis. The details of what is covered in the classroom are tailored to the level of the students. For well-motivated students, very little time is spent on syntax, and the course turns out to be very close to the previously described parallel algorithms class. For less motivated students, I run the class in much the same way as an ``Introduction to Programming Class.'' Each week or two I introduce a new problem and then spend a week or two discussing the development of a solution. This typically involves fairly extensive discussions of the syntax and semantics of MPI functions.
I also teach a class for seniors and graduate students in which they spend a year working on a parallel programming project sponsored by a company or government agency. In this class, I usually spend a week or two on an overview of parallel computing. The students learn parallel programming by working through PPMPI on their own.