SW:moose

Build MOOSE framework in user scratch

   cd $SCRATCH
   module load MOOSE/moose-dev-gcc-ompi
   $EBROOTMOOSE/install-moose.sh

It will take a while for the installation to finish. When it is done, the moose framework will be install in your current directory. Now you can build your model with the moose framework. Do as follows.

1. Copy the your module to /path/to/moose/modules

   cp -R /path/to/my_module /path/to/moose/modules

2. Make sure moose-dev-gcc-ompi is loaded

   module load MOOSE/moose-dev-gcc-ompi  

3. Build your module

   cd /path/to/moose/modules/my_module
   make

Sample job scripts on Terra

Sample 1

   #!/bin/bash
   ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION
   #SBATCH --get-user-env=L       #Replicate login environment
   #SBATCH -J moose-sample1       #Set the job name to "moose-sample1"
   #SBATCH -t 1:00:00             #Set the wall clock limit to 1hr 
   #SBATCH -N 20                  #Request 20 node
   #SBATCH --ntasks-per-node=28   #Request 28 task/core per node
   #SBATCH --mem=57000M           #Request 57G per node
   #SBATCH -o moose-sample1.%j    #Send stdout/err to "Example1Out.[jobID]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun /path/to/moose-opt -i moose.i

Sample 2

   #!/bin/bash
   ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION
   #SBATCH --get-user-env=L       #Replicate login environment   
   #SBATCH -J  moose-sample2      #Set the job name to "moose-sample2"
   #SBATCH -t 1:00:00             #Set the wall clock limit to 1hr 
   #SBATCH -N 20                  #Request 20 node
   #SBATCH --ntasks-per-node=10   #Request 10 task/core per node
   #SBATCH --mem=58=7000M         #Request 57G per node
   #SBATCH -o moose-sample2.%j    #Send stdout/err to "moose-sample2.[jobID]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun -np 200 -npernode 10 /path/to/moose-opt -i moose.i

Sample job script on Ada

Sample 1

   #BSUB -J moose-sample1
   #BSUB -o moose-sample1.%J
   #BSUB -e error.%J
   #BSUB -L /bin/bash
   #BSUB -W 20:00
   #BSUB -n 400
   #BSUB -M 2700
   #BSUB -R "span[ptile=20]"
   #BSUB -R "rusage[mem=2700]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun /path/to/moose-opt -i moose.i

Sample 2: Use half of the cores per node such that each core can have 2700x2=5400M memory to use. This is useful for models that need large amount of memory.

   #BSUB -J moose-sample2
   #BSUB -o moose-sample2.%J
   #BSUB -e error.%J
   #BSUB -L /bin/bash
   #BSUB -W 20:00
   #BSUB -n 400
   #BSUB -M 2700
   #BSUB -R "span[ptile=20]"
   #BSUB -R "rusage[mem=2700]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun -np 200 -npernode 10 /path/to/moose-opt -i moose.i