Curie:Compile:MPI

MPI programs

OpenMPI

Curie has multiple MPI stacks installed (mpich, mpich2, openmpi). To see a complete list type "module spider mpi". In this README file we will use OpenMPI but the instructions also apply to any of the other mpi stacks (there might be a few exceptions). To load the latest OpenMPI stack (including xlf compilers) type "module load xlompi".

To compile you will need to use the appropriate wrapper:

MPI wrapper	Commands
Fortran wrapper	mpifort
C compilers	mpicc
C++ compilers	mpic++

NOTE: these wrappers use the non-thread safe compilers underneath (i.e. without the "_r" extension). We will provide thread safe wrappers asap.

To see the full compiler command for any of the mpi wrappers use the "-show flag" (e.g. mpifort -show). This will show a full listing including compiler flags, LIBRARY paths, and INCLUDE paths and might be useful for debugging purposes.

Example: compile MPI program hello_mpi.c and name if hello_mpi.x

mpicc -o hello_mpi.x hello_mpi.c

Running MPI code

Unlike running serial or OpenMP code, running mpi code requires a launcher. The launcher will set up the environment and start the requested number of tasks. On curie the launcher is named "mpirun" and has the following syntax:

mpirun [mpi_flags] <executable> [exec params]

The following table shows some of the more common flags (for a full listing type "man mpirun")

Flags	Descriptions
-np	number of tasks to start
-hosts <list>	comma separated list of hostsnames to start tasks on
-hostfile <file>	specifies hostfile (containing list of hosts to use)
-map-by ppr:N:node	places N processes on each node.

Example: run the program hello_mpi.x using 8 tasks, 4 on curie1, 4 on curie2.

mpirun -np 8 -hosts curie1,curie2 -map-by ppr:4:node ./hello_mpi.x