Curie:Compile:All

Getting Started

Before you can use any of the installed software packages (e.g. compilers, libraries) you will need to setup the correct environment (e.g. set PATH, LD_LIBRARY_PATH, etc). On curie this is done using the "module" system.

To load a module use: module load PACKAGE, where PACKAGE is the name of the software you want to setup. For example, to use the xlf compiler type, module load xlf. To see all available packages type the module spider command. To see all installed versions for a specific package type, module spider PACKAGE.

To see a list of all installed packages type: module list For more information about modules, please visit, https://hprc.tamu.edu/wiki/Curie:Computing_Environment#Modules

Compiling

The preferred compilers on curie are the IBM XL Compilers. The table below shows the various compilers:

Compilers	Commands
Fortran compilers	xlf/xlf_r
C compilers	xlc/xlc_r
C++ compilers	xlc++/xlc++_r

The compilers with the "_r" extension will generate threadsafe code and are used to compile threaded programs (e.g., programs containing OpenMP directives/constructs)

By default, the XL compilers generate 32-bit code. To generate 64-bit code you can use the -q64 compiler flag or set the environmental variable OBJECT_MODE to 64 (NOTE, when you load any of the XL compiler modules OBJECT_MODE is set to 64 automatically). Curie:Compile:Scalar Curie:Compile:OpenMP

MPI programs

OpenMPI

Curie has multiple MPI stacks installed (mpich, mpich2, openmpi). To see a complete list type "module spider mpi". In this README file we will use OpenMPI but the instructions also apply to any of the other mpi stacks (there might be a few exceptions). To load the latest OpenMPI stack (including xlf compilers) type "module load xlompi".

To compile you will need to use the appropriate wrapper:

MPI wrapper	Commands
Fortran wrapper	mpifort
C compilers	mpicc
C++ compilers	mpic++

NOTE: these wrappers use the non-thread safe compilers underneath (i.e. without the "_r" extension). We will provide thread safe wrappers asap.

To see the full compiler command for any of the mpi wrappers use the "-show flag" (e.g. mpifort -show). This will show a full listing including compiler flags, LIBRARY paths, and INCLUDE paths and might be useful for debugging purposes.

Example: compile MPI program hello_mpi.c and name if hello_mpi.x

mpicc -o hello_mpi.x hello_mpi.c

Running MPI code

Unlike running serial or OpenMP code, running mpi code requires a launcher. The launcher will set up the environment and start the requested number of tasks. On curie the launcher is named "mpirun" and has the following syntax:

mpirun [mpi_flags] <executable> [exec params]

The following table shows some of the more common flags (for a full listing type "man mpirun")

Flags	Descriptions
-np	number of tasks to start
-hosts <list>	comma separated list of hostsnames to start tasks on
-hostfile <file>	specifies hostfile (containing list of hosts to use)
-map-by ppr:N:node	places N processes on each node.

Example: run the program hello_mpi.x using 8 tasks, 4 on curie1, 4 on curie2.

mpirun -np 8 -hosts curie1,curie2 -map-by ppr:4:node ./hello_mpi.x