Hprc banner tamu.png

Difference between revisions of "SW:moose"

From TAMU HPRC
Jump to: navigation, search
(Building the MOOSE framework in your $SCRATCH)
(Testing your MOOSE framework)
Line 13: Line 13:
  
 
== Testing your MOOSE framework ==
 
== Testing your MOOSE framework ==
The first tests to run were built at the end of the build process above.  To run them, do:
 
 
  ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
 
  cd $SCRATCH/moose/test
 
  ./run_tests
 
 
That will run for nearly an hour (on terra) and will have output that ends in something like:
 
 
    time_integrators/convergence.implicit_astabledirk4_bootstrap/level1 ....................................... OK
 
    geomsearch/2d_moving_penetration.restart .................................................................. OK
 
    time_integrators/convergence.implicit_astabledirk4_bootstrap/level2 ....................................... OK
 
    geomsearch/2d_moving_penetration.restart2 ................................................................. OK
 
    samplers/distribute.scale/execute .................................................. [FINISHED] FAILED (CRASH)
 
    --------------------------------------------------------------------------------------------------------------
 
    Ran 2331 tests in 2768.0 seconds.
 
    2330 passed, 84 skipped, 0 pending, 1 FAILED
 
 
[Note: The failure is left as an exercise to the user if it matters]
 
 
  
 
At this point, to do further testing of your installation you can build all the exercises with:
 
At this point, to do further testing of your installation you can build all the exercises with:
Line 37: Line 18:
 
     ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
 
     ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
 
     cd $SCRATCH/moose/examples
 
     cd $SCRATCH/moose/examples
     make -j 4 # build
+
     make -j 8 # build
  
 
For descriptions, and instructions on how to run the examples, see the [https://mooseframework.org/examples/index.html MOOSE Examples] page.
 
For descriptions, and instructions on how to run the examples, see the [https://mooseframework.org/examples/index.html MOOSE Examples] page.
Line 45: Line 26:
 
     ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
 
     ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
 
     cd $SCRATCH/moose/modules
 
     cd $SCRATCH/moose/modules
     make -j 4 # takes a bit of time
+
     make -j 8 # takes a bit of time
     ./run_tests # takes a long time
+
     ./run_tests -j 8 # takes a long time
  
 
== Using your MOOSE framework ==
 
== Using your MOOSE framework ==

Revision as of 13:52, 22 July 2020

Multiphysics Object-Oriented Simulation Environment - An open-source, parallel finite element framework

Building the MOOSE framework in your $SCRATCH

   cd $SCRATCH # where to install
   module purge # clear module environment
   module load MOOSE/moose-dev-gcc-ompi # load the MOOSE development module
   time $EBROOTMOOSE/install-moose.sh build # get/build/test MOOSE

This will download and build the MOOSE framework in $SCRATCH/moose. It runs a script called update_and_rebuild_libmesh.sh that takes the majority of the time. After that, it will switch to the $SCRATCH/moose/test directory and run 'make' and 'run_tests' to finish the build.

Testing your MOOSE framework

At this point, to do further testing of your installation you can build all the exercises with:

   ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
   cd $SCRATCH/moose/examples
   make -j 8 # build

For descriptions, and instructions on how to run the examples, see the MOOSE Examples page.

Do do some very extensive, and lengthy, testing, you can build all the MOOSE modules and run their tests.

   ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
   cd $SCRATCH/moose/modules
   make -j 8 # takes a bit of time
   ./run_tests -j 8 # takes a long time

Using your MOOSE framework

(this needs more work... see the MOOSE web page for more details after the framework is installed)

Now you can build your model with the moose framework. Do as follows.

   ml MOOSE/moose-dev-gcc-ompi # make sure module is loaded
   cp -rp /path/to/my_module $SCRATCH/moose/modules # copy your module to framework
   cd $SCRATCH/moose/modules/my_module
   # MIGHT NEED A STEP HERE TO USE MOOSE TO CREATE A Makefile
   make

Sample job scripts on Terra

Sample 1

   #!/bin/bash
   ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION
   #SBATCH --get-user-env=L       #Replicate login environment
   #SBATCH -J moose-sample1       #Set the job name to "moose-sample1"
   #SBATCH -t 1:00:00             #Set the wall clock limit to 1hr 
   #SBATCH -N 20                  #Request 20 node
   #SBATCH --ntasks-per-node=28   #Request 28 tasks per node
   #SBATCH --mem=56G              #Request 56G per node
   #SBATCH -o moose-sample1.%j    #Send stdout/err to "Example1Out.[jobID]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun /path/to/moose-opt -i moose.i

Sample 2 Use half of the cores per node such that each core can have 2G*2=4G memory to use. This is useful for models that need large amount of memory.

   #!/bin/bash
   ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION
   #SBATCH --get-user-env=L       #Replicate login environment   
   #SBATCH -J  moose-sample2      #Set the job name to "moose-sample2"
   #SBATCH -t 1:00:00             #Set the wall clock limit to 1hr 
   #SBATCH -N 20                  #Request 20 node
   #SBATCH --ntasks-per-node=28   #Request 28 tasks per node
   #SBATCH --mem=56G              #Request 56G per node
   #SBATCH -o moose-sample2.%j    #Send stdout/err to "moose-sample2.[jobID]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun -np 280 -npernode 14 /path/to/moose-opt -i moose.i

Sample job script on Ada

Sample 1

   #BSUB -J moose-sample1
   #BSUB -o moose-sample1.%J
   #BSUB -e error.%J
   #BSUB -L /bin/bash
   #BSUB -W 20:00
   #BSUB -n 400
   #BSUB -M 2700
   #BSUB -R "span[ptile=20]"
   #BSUB -R "rusage[mem=2700]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun /path/to/moose-opt -i moose.i

Sample 2: Use half of the cores per node such that each core can have 2700x2=5400M memory to use. This is useful for models that need large amount of memory.

   #BSUB -J moose-sample2
   #BSUB -o moose-sample2.%J
   #BSUB -e error.%J
   #BSUB -L /bin/bash
   #BSUB -W 20:00
   #BSUB -n 400
   #BSUB -M 2700
   #BSUB -R "span[ptile=20]"
   #BSUB -R "rusage[mem=2700]"
   
   module load MOOSE/moose-dev-gcc-ompi
   mpirun -np 200 -npernode 10 /path/to/moose-opt -i moose.i