Difference between revisions of "SW:ABAQUS"
|Line 76:||Line 76:|
To submit the batch job, run:
To submit the batch job, run:
[NetID@ada1 ~]$ '''bsub < ''jobscript'''''
[ NetID@ada1 ~]$ '''bsub < ''jobscript'''''
Revision as of 12:33, 24 March 2017
Finite Element Analysis software for modeling, visualization, and best-in-class implicit and explicit dynamics FEA. - Homepage: http://www.3ds.com/products-services/simulia/products/abaqus/
We have documentation available for the following versions of ABAQUS:
Note: You will need to be within the TAMU firewall in order to view these documents. You will need to either be on campus OR connect using VPN. Information on the TAMU VPN can be found here.
Abaqus is open to all HPRC users when used within the terms of our license agreement. If you have particular concerns about whether specific usage falls within the TAMU HPRC license, please send an email to the HPRC Helpdesk.
Loading the Module
To see all versions of Abaqus available on Ada:
[NetID@cluster ~]$ module spider ABAQUS
To load a particular version of Abaqus use: (Example: 6.13.5)
[NetID@cluster ~]$ module load ABAQUS/6.13.5-linux-x86_64
Note: New versions of software become available periodically. Version numbers may change.
Usage of Abaqus is restricted by the number of available tokens. One instance of Abaqus cae requires one (1) cae token. An Abaqus job requires tokens based on the number of cores the job will use. The formula for calculating licenses that are needed is:
int(5 * (cores)^0.422)
To see the number of available tokens, use the License Checker Tool.
Usage on the Login Nodes
Please limit interactive processing to short, non-intensive usage. Use non-interactive batch jobs for resource-intensive and/or multiple-core processing. Users are requested to be responsible and courteous to other users when using software on the login nodes.
The most important processing limits here are:
- ONE HOUR of PROCESSING TIME per login session.
- EIGHT CORES per login session on the same node or (cumulatively) across all login nodes.
Anyone found violating the processing limits will have their processes killed without warning. Repeated violation of these limits will result in account suspension.
Note: Your login session will disconnect after one hour of inactivity.
Usage on the Compute Nodes
Non-interactive batch jobs on the compute nodes allows for resource-demanding processing. Non-interactive jobs have higher limits on the number of cores, amount of memory, and runtime length.
For instructions on how to create and submit a batch job, please see the appropriate wiki page for each respective cluster:
A single core example, with user subroutine: (Last updated Dec. 9, 2016)
#BSUB -J AbaqusJob # sets the job name to AbaqusJob. #BSUB -L /bin/bash # uses the bash login shell to initialize the job's execution environment. #BSUB -W 2:00 # sets to 2 hours the job's runtime wall-clock limit. #BSUB -n 1 # assigns 1 core for execution. #BSUB -R "span[ptile=1]" # assigns 1 core per node. #BSUB -R "rusage[mem=5000]" # reserves 5000MB per process/CPU for the job (5GB * 1 Core = 5GB per node) #BSUB -M 5000 # sets to 5,000MB (~5GB) the per process enforceable memory limit. #BSUB -o Abaqus.%J # directs the job's standard output to Abaqus.jobid ## Load the modules module load intel/2016b module load ABAQUS/6.14.2-linux-x86_64 ## Launch Abaqus with proper parameters abaqus memory="5GB" cpus=1 job=jobname input=inputfile.inp user=filename.for
A multicore (10 core) example, with no user subroutine: (Last updated Dec. 9, 2016)
#BSUB -J AbaqusJob # sets the job name to AbaqusJob. #BSUB -L /bin/bash # uses the bash login shell to initialize the job's execution environment. #BSUB -W 2:00 # sets to 2 hours the job's runtime wall-clock limit. #BSUB -n 10 # assigns 10 cores for execution. #BSUB -R "span[ptile=10]" # assigns 10 cores per node. #BSUB -R "rusage[mem=5000]" # reserves 5000MB per process/CPU for the job (5GB * 10 Core = 50GB per node) #BSUB -M 5000 # sets to 5,000MB (~5GB) the per process enforceable memory limit. #BSUB -o Abaqus.%J # directs the job's standard output to Abaqus.jobid ## Load the module module load ABAQUS/6.14.2-linux-x86_64 ## Launch Abaqus with proper parameters abaqus memory="50GB" cpus=10 job=jobname input=inputfile.inp mp_mode=mpi
To submit the batch job, run: (where jobscript looks like one of the above examples)
[ NetID@ada1 ~]$ bsub < jobscript
There is a known error with Abaqus versions prior to 2017 on Terra that requires unsetting a Slurm environment variable. To prevent this error, please add the following line to your job script before the abaqus launch command.
A single core example with a user subroutine: (Last updated March 24, 2017)
#!/bin/bash ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION #SBATCH --export=NONE #SBATCH --get-user-env=L ##NECESSARY JOB SPECIFICATIONS #SBATCH --job-name=AbaqusJob # Sets the job name to AbaqusJob #SBATCH --time=2:00:00 # Sets the runtime limit to 2 hr #SBATCH --ntasks=1 # Requests 1 core #SBATCH --ntasks-per-node=1 # Requests 1 core per node (1 node) #SBATCH --mem=5G # Requests 5GB of memory per node #SBATCH --output=AbaqusJob.o%J # Sends stdout and stderr to AbaqusJob.o[jobID] ## Load the module module purge module load intel/2017A module load ABAQUS/2017 ## Launch Abaqus with proper parameters abaqus memory="5GB" cpus=10 job=jobname input=inputfile.inp user=filename.for
A multicore (10 core) example, with no user subroutine: (Last updated March 24, 2017)
#!/bin/bash ##ENVIRONMENT SETTINGS; CHANGE WITH CAUTION #SBATCH --export=NONE #SBATCH --get-user-env=L ##NECESSARY JOB SPECIFICATIONS #SBATCH --job-name=AbaqusJob # Sets the job name to AbaqusJob #SBATCH --time=2:00:00 # Sets the runtime limit to 2 hr #SBATCH --ntasks=10 # Requests 10 cores #SBATCH --ntasks-per-node=10 # Requests 10 cores per node (1 node) #SBATCH --mem=50G # Requests 50GB of memory per node #SBATCH --output=AbaqusJob.o%J # Sends stdout and stderr to AbaqusJob.o[jobID] ## Load the module module purge module load ABAQUS/2017 ## Launch Abaqus with proper parameters abaqus memory="50GB" cpus=10 job=jobname input=inputfile.inp mp_mode=mpi
To submit the batch job run: (where jobscript is a file that looks like one of the above examples)
[ NetID@terra ~]$ sbatch jobscript
Usage on the VNC Nodes
The VNC nodes allow for usage of the a graphical user interface (GUI) without disrupting other users.
VNC jobs and GUI usage do come with restrictions. All VNC jobs are limited to a single node (Terra: 28 cores/64GB). There are fewer VNC nodes than comparable compute nodes.
For more information, including instructions, on using software on the VNC nodes, please visit our Terra Remote Visualization page.
Running the ABAQUS GUI
While in a VNC job, use abaqus cae (with vglrun) to start the ABAQUS GUI:
[NetID@gpu ~]$ vglrun abaqus cae
Frequently Asked Questions
Q: What versions of ABAQUS are available and which should I use?
A: You can see which versions of ABAQUS we have available by using the module spider command as shown above. If you do not need to use a specific version of ABAQUS, we generally recommend that you use the newest version that we provide. Remember that it is always recommended to load a specific version of a module instead of the default version. This is because default versions will change which might cause problems in the future. It is always best to know exactly which module you are using.
Q: How do I open the ABAQUS GUI?
A: Using the ABAQUS GUI (or almost any GUI on our clusters) for anything more than light editing requires the use of a VNC job. For information on how to use a VNC job, please see our Remote Visualization page.
- Step 1: Start a VNC session with the instructions from the page linked above.
- Step 2: Load the ABAQUS module as shown above.
- Step 3: Launch the ABAQUS GUI as shown below:
[NetID@gpu ~]$ vglrun abaqus cae
Note: We always recommend that you do not submit jobs with the ABAQUS GUI unless they take only seconds to run. Using the GUI to submit jobs can cause a lot of issues and often takes much longer than running a job non-interactively.
Q: What is the difference between an interactive and a non-interactive job?
A: An interactive job is one which is done in the GUI with interaction from the user (even if only to start the job). A non-interactive job is a job that the user gives to the cluster, most often with a job file, which is then scheduled and run by the batch system without the GUI or any further interaction from the user. A non-interactive job often runs faster while using less resources and can more easily be customized to meet the user's needs. A non-interactive job uses the ABAQUS solver and creates exactly the same files as it would if the GUI were used. You will also still be able to view the results of the analysis as though the job were run with the GUI.
Q: How do I submit a non-interactive job?
A: You will need a batch job file and your ABAQUS input file (.inp) in order to submit a job. Examples of ABAQUS job files are given above. Please use these only as guidelines, you will need to customize some requirements to meet your needs.
- Step 1: Generate the ABAQUS input file (.inp). This can be done in the GUI by first creating a job and then, in the top toolbar, going to Job->Write Input->JobName.
- Step 2: Create a batch job file with the appropriate parameters.
- Step 3: Submit the job
Use bsub on Ada:
[NetID@ada1 ~]$ bsub < jobscript
Use sbatch on Terra:
[NetID@terra1 ~]$ sbatch jobscript
Q: How much memory do I need?
A: ABAQUS has a datacheck feature which will estimate how much memory is needed to complete a certain job.
- Step 1: Load the ABAQUS module.
- Step 2: Run the datacheck command with the proper parameters. A data file (.dat) will be created.
[NetID@cluster ~]$ abaqus datacheck job=jobname
- Step 3: Look in the (.dat) file, there will be a section titled "MEMORY ESTIMATE" as shown below. The value labled "MEMORY TO MINIMIZE I/O" is the total amount of memory that your job will need. This is only an estimate and it is recommended to request at least slightly over the value found here.
M E M O R Y E S T I M A T E PROCESS FLOATING PT MINIMUM MEMORY MEMORY TO OPERATIONS REQUIRED MINIMIZE I/O PER ITERATION (MBYTES) (MBYTES) 1 4.60E+6 10336 75311
Q: How do I set the memory?
A: There are two main ways that you can set the memory for ABAQUS.
- Setting the memory for an ABAQUS job. (Recommended for use with ALL non interactive jobs)
- To use more than the default memory in an ABAQUS job, add the memory parameter to your ABAQUS launch command in your batch job file. Remember, this is TOTAL memory, not memory per core.
abaqus memory="5000mb" job=jobname input=inputfile
- Setting the memory in an ABAQUS environment file. (Recommended for use with the GUI)
- In order to use more memory with the ABAQUS GUI, you will need to create an ABAQUS environment file (.env) which can be placed in either your home directory or current working directory.
- Step 1: Create an ABAQUS environment file titled abaqus_v6.env in your $HOME directory or the current working directory (or both).
- Step 2: Add an entry to the environment file for memory with the memory parameter. Remember this is TOTAL memory, not memory per core.
Note: ABAQUS checks for environment settings in three places. In order, these are: the installation directory, a user's HOME directory, and the current working directory. The settings that take precedence are the LAST settings that ABAQUS checks. For example: If you have an environment file in your home directory AND the current working directory, the settings specified in the current working directory are the ones that will be used. Any settings specified in an ABAQUS launch command will take precedence over all environment file settings. If no memory is specified, the default value is 2GB.
Q: How many cores should I use?
A: ABAQUS is a software that is not the best when it comes to parallelization. Unless you have written your code specifically to be parallelized, ABAQUS generally will not parallelize well. It is possible that you could experience some speed up from running your job on multiple cores, and it is worth trying if your code is thread-safe. However, if you are running a small job (which takes less than 2 hours to complete), it is probably not worth the trouble of submitting on multiple cores. If you have a job that will run for longer, you might see a significant difference when using multiple cores. In general, for smaller jobs, use fewer cores, and for larger jobs, use more cores.
Note: Do not try to run your job on multiple cores if ANY part of your code is not thread safe.
Q: How long will my job take?
If you are using a single core, in general, you can expect that an ABAQUS job on Ada will take approximately the same time as it would on a regular workstation. As stated above, ABAQUS generally does not parallelize well. If you are using multiple cores, you might experience some speed up, but it is hard to say how much as it depends largely on your job and code.
Q: How do I use ABAQUS to do [blank]?
A: Although we have some experience with running ABAQUS, we are, for the most part, NOT ABAQUS experts. We can only provide minimal help when it comes to using the software. If you have a problem with the software not behaving how you expect it to, please refer to the ABAQUS documentation linked above. Please feel free to contact us with your questions, but keep in mind that we are NOT ABAQUS experts.