Hprc banner tamu.png

Difference between revisions of "SW:moose"

From TAMU HPRC
Jump to: navigation, search
(Testing your MOOSE framework)
(Using your MOOSE framework)
Line 25: Line 25:
 
Now you can build your model with the moose framework. Do as follows.
 
Now you can build your model with the moose framework. Do as follows.
  
1. Copy your module to /path/to/moose/modules
+
     cp -rp /path/to/my_module $SCRATCH/moose/modules
 
 
     cp -R /path/to/my_module /path/to/moose/modules
 
 
 
2.  Make sure moose-dev-gcc-ompi is loaded
 
 
 
 
     module load MOOSE/moose-dev-gcc-ompi   
 
     module load MOOSE/moose-dev-gcc-ompi   
 
+
     cd $SCRATCH/moose/modules/my_module
3. Build your module
+
    # MIGHT NEED A STEP HERE TO USE MOOSE TO CREATE A Makefile
 
 
     cd /path/to/moose/modules/my_module
 
 
     make
 
     make
  

Revision as of 17:59, 21 July 2020

Building the MOOSE framework in your $SCRATCH

   cd $SCRATCH
   module load MOOSE/moose-dev-gcc-ompi
   $EBROOTMOOSE/install-moose.sh # get sources, update libmesh, finish build

This will download and build the MOOSE framework in $SCRATCH/moose. It runs a script called update_and_rebuild_libmesh.sh that takes 3-4 hours to build. After that, it will switch to the $SCRATCH/moose/test directory and run 'make' there to finish the build.

Testing your MOOSE framework

At this point, to do basic testing of your installation you can build all the exercises with:

   ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
   cd $SCRATCH/moose/examples
   make -j 4 # build

For descriptions and instructions on how to run see the MOOSE Examples page.

Using your MOOSE framework

(this needs more work... see the MOOSE web page for more details after the framework is installed)

Now you can build your model with the moose framework. Do as follows.

   cp -rp /path/to/my_module $SCRATCH/moose/modules
   module load MOOSE/moose-dev-gcc-ompi  
   cd $SCRATCH/moose/modules/my_module
   # MIGHT NEED A STEP HERE TO USE MOOSE TO CREATE A Makefile
   make

Sample job scripts on Terra

Sample 1

   #!/bin/bash
   ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION
   #SBATCH --get-user-env=L       #Replicate login environment
   #SBATCH -J moose-sample1       #Set the job name to "moose-sample1"
   #SBATCH -t 1:00:00             #Set the wall clock limit to 1hr 
   #SBATCH -N 20                  #Request 20 node
   #SBATCH --ntasks-per-node=28   #Request 28 tasks per node
   #SBATCH --mem=56G              #Request 56G per node
   #SBATCH -o moose-sample1.%j    #Send stdout/err to "Example1Out.[jobID]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun /path/to/moose-opt -i moose.i

Sample 2 Use half of the cores per node such that each core can have 2G*2=4G memory to use. This is useful for models that need large amount of memory.

   #!/bin/bash
   ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION
   #SBATCH --get-user-env=L       #Replicate login environment   
   #SBATCH -J  moose-sample2      #Set the job name to "moose-sample2"
   #SBATCH -t 1:00:00             #Set the wall clock limit to 1hr 
   #SBATCH -N 20                  #Request 20 node
   #SBATCH --ntasks-per-node=28   #Request 28 tasks per node
   #SBATCH --mem=56G              #Request 56G per node
   #SBATCH -o moose-sample2.%j    #Send stdout/err to "moose-sample2.[jobID]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun -np 280 -npernode 14 /path/to/moose-opt -i moose.i

Sample job script on Ada

Sample 1

   #BSUB -J moose-sample1
   #BSUB -o moose-sample1.%J
   #BSUB -e error.%J
   #BSUB -L /bin/bash
   #BSUB -W 20:00
   #BSUB -n 400
   #BSUB -M 2700
   #BSUB -R "span[ptile=20]"
   #BSUB -R "rusage[mem=2700]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun /path/to/moose-opt -i moose.i

Sample 2: Use half of the cores per node such that each core can have 2700x2=5400M memory to use. This is useful for models that need large amount of memory.

   #BSUB -J moose-sample2
   #BSUB -o moose-sample2.%J
   #BSUB -e error.%J
   #BSUB -L /bin/bash
   #BSUB -W 20:00
   #BSUB -n 400
   #BSUB -M 2700
   #BSUB -R "span[ptile=20]"
   #BSUB -R "rusage[mem=2700]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun -np 200 -npernode 10 /path/to/moose-opt -i moose.i