Curie has multiple MPI stacks installed (mpich, mpich2, openmpi). To see a complete list type "module spider mpi". In this README file we will use OpenMPI but the instructions also apply to any of the other mpi stacks (there might be a few exceptions). To load the latest OpenMPI stack (including xlf compilers) type "module load xlompi".
To compile you will need to use the appropriate wrapper:
NOTE: these wrappers use the non-thread safe compilers underneath (i.e. without the "_r" extension). We will provide thread safe wrappers asap.
To see the full compiler command for any of the mpi wrappers use the "-show flag" (e.g. mpifort -show). This will show a full listing including compiler flags, LIBRARY paths, and INCLUDE paths and might be useful for debugging purposes.
Example: compile MPI program hello_mpi.c and name if hello_mpi.x
mpicc -o hello_mpi.x hello_mpi.c
Running MPI code
Unlike running serial or OpenMP code, running mpi code requires a launcher. The launcher will set up the environment and start the requested number of tasks. On curie the launcher is named "mpirun" and has the following syntax:
mpirun [mpi_flags] <executable> [exec params]
The following table shows some of the more common flags (for a full listing type "man mpirun")
|-np||number of tasks to start|
|-hosts <list>||comma separated list of hostsnames to start tasks on|
|-hostfile <file>||specifies hostfile (containing list of hosts to use)|
|-map-by ppr:N:node||places N processes on each node.|
Example: run the program hello_mpi.x using 8 tasks, 4 on curie1, 4 on curie2.
mpirun -np 8 -hosts curie1,curie2 -map-by ppr:4:node ./hello_mpi.x