MPI Usage Information

MPI requires a wrapper (mpicc) to compile applications. You’ll also need to use mpirun (or one of its many variants) to run MPI applications.

Basic Compile and Run

# This compiles an MPI source file called 'hello.c'. The output binary
# will be named 'hello.'
$ mpicc -g -Wall hello.c -o hello

# This will run the MPI application using the default number of processes
# on your machine. Mine, for example, has four cores so four processes are
# run.
$ mpirun ./hello
Hello! (rank 0)
Hello! (rank 3)
Hello! (rank 1)
Hello! (rank 2)

Note that when you run an MPI application and print to the terminal (stdout), we don’t have control over the order that things get printed.

Setting the Number of Processes

Occasionally we will want to use something other than the defaults when running an MPI application. Do do this, use the -n command line flag. If you want to run more processes than the number of cores available, use --oversubscribe:

$ mpirun -n 1 ./hello
Hello! (rank 0)

$ mpirun -n 25 --oversubscribe ./hello
Hello! (rank 0)
Hello! (rank 22)
Hello! (rank 3)
Hello! (rank 14)
Hello! (rank 12)

(output clipped)

Running on Multiple Machines

To run your MPI jobs across several machines, you will need a ‘hostfile’ – a plain text list of the machines to run on.

Generating the Hostfile

If you have several numbered machines you’d like to use, you can use the following bash shell snippet (in your terminal) to generate the list:

$ for i in {01..24}; do echo "jet${i}"; done > hostfile.txt

# Let's look at the first few entries in the text file we created:
$ head -n 3 hostfile.txt
jet01
jet02
jet03

Otherwise, you can also edit a list of machines by hand. Once you have your hostfile ready, you can pass it to mpirun.

Running With a Hostfile

# Runs 24 processes across the hosts found in hostfile.txt:
$ mpirun --hostfile ./hostfile.txt -n 24 ./hello

# Runs 96 processes across the hosts in hostfile.txt, ensuring each host 
# manages four processes:
$ mpirun -n 96 --hostfile ./hostfile.txt --npersocket 4 ./hello

# And finally, oversubscribing with multiple machines:
$ mpirun -n 192 --hostfile ./hostfile.txt --npersocket 4 --oversubscribe ./hello