Hprc banner tamu.png

Difference between revisions of "Ada:QuickStart"

From TAMU HPRC
Jump to: navigation, search
(Running Your Program / Preparing a Job File)
(Navigating Ada & Storage Quotas)
(40 intermediate revisions by 8 users not shown)
Line 4: Line 4:
 
'''Access to Ada is granted with the condition that you will understand and adhere to all TAMU HPRC and Ada-specific policies.'''  
 
'''Access to Ada is granted with the condition that you will understand and adhere to all TAMU HPRC and Ada-specific policies.'''  
  
General policies can be found on the [[HPRC:Policies | HPRC Policies page]].
+
General policies can be found on the [https://hprc.tamu.edu/policies/ HPRC Policies page].
  
Ada-specific policies, which are similar to Terra, can be found on the [[Ada:Policies | Ada Policies page]].  
+
Ada-specific policies can be found on the [[Ada:Policies | Ada Policies page]].
  
 
== Accessing Ada ==
 
== Accessing Ada ==
  
Most access to Ada is done via a secure shell session.
 
  
Users on '''Windows''' computers use either [http://www.putty.org/ PuTTY] or [http://mobaxterm.mobatek.net/ MobaXterm]. If MobaXterm works on your computer, it is usually easier to use.
+
<!--'''For convenience, this topic has been summarized in a short video lesson, which you can view [https://www.youtube.com/watch?v=dypaj5uHpqQ here]. Otherwise, feel free to continue reading.'''-->
 +
 
 +
<!--Most access to Ada is done via a secure shell session.
 +
 
 +
Users on '''Windows''' computers use either [http://mobaxterm.mobatek.net/ MobaXterm] or [http://www.putty.org/ PuTTY]. If MobaXterm works on your computer, it is usually easier to use.
  
 
Users on '''Mac''' and '''Linux/Unix''' should use whatever SSH-capable terminal is available on their system.
 
Users on '''Mac''' and '''Linux/Unix''' should use whatever SSH-capable terminal is available on their system.
  
 
The command to connect to Ada is as follows. Be sure to replace [NetID] with your TAMU NetID.  
 
The command to connect to Ada is as follows. Be sure to replace [NetID] with your TAMU NetID.  
 +
[user1@localhost ~]$ '''ssh ''NetID''@ada.tamu.edu'''
 +
<font color=teal>'''Note:''' In this example ''[user1@localhost ~]$'' represents the command prompt on your local machine.</font> <br>
 +
Your login password is the same that used on [https://howdy.tamu.edu/ Howdy]. You will not see your password as your type it into the login prompt.-->
 +
 +
Most access to Ada is done via a secure shell session. In addition, '''two-factor authentication''' is required to login to any cluster.
 +
 +
Users on '''Windows''' computers use either [http://www.putty.org/ PuTTY] or [http://mobaxterm.mobatek.net/ MobaXterm]. If MobaXterm works on your computer, it is usually easier to use. When starting an ssh session in PuTTY, choose the connection type 'SSH', select port 22, and then type the hostname 'ada.tamu.edu'. For MobaXterm, select 'Session', 'SSH', and then remote host 'ada.tamu.edu'. Check the box to specify username and type your NetID. After selecting 'Ok', you will be prompted for Duo Two Factor Authentication. For more detailed instructions, visit the [https://hprc.tamu.edu/wiki/Two_Factor#MobaXterm Two Factor Authentication] page.
 +
 +
Users on '''Mac''' and '''Linux/Unix''' should use whatever SSH-capable terminal is available on their system. The command to connect to Ada is as follows. Be sure to replace [NetID] with your TAMU NetID.
 
  [user1@localhost ~]$ '''ssh ''[NetID]''@ada.tamu.edu'''
 
  [user1@localhost ~]$ '''ssh ''[NetID]''@ada.tamu.edu'''
 
<font color=teal>'''Note:''' In this example ''[user1@localhost ~]$'' represents the command prompt on your local machine.</font> <br>
 
<font color=teal>'''Note:''' In this example ''[user1@localhost ~]$'' represents the command prompt on your local machine.</font> <br>
 
Your login password is the same that used on [https://howdy.tamu.edu/ Howdy]. You will not see your password as your type it into the login prompt.
 
Your login password is the same that used on [https://howdy.tamu.edu/ Howdy]. You will not see your password as your type it into the login prompt.
 +
 +
=== Off Campus Access ===
 +
Please visit [https://hprc.tamu.edu/wiki/HPRC:Remote_Access this page] to find information on accessing Ada remotely.
 +
 +
For more detailed instructions on how to access our systems, please see the [[HPRC:Access | HPRC Access page]].
  
 
== Navigating Ada & Storage Quotas ==
 
== Navigating Ada & Storage Quotas ==
 +
When users first access Ada, they will be within their ''home'' directory. This directory has smaller storage quotas and should not be used for general purpose.
  
When you first access Ada, you will be within your ''home'' directory. This directory has smaller storage quotas and should not be used for general purpose.
+
Users can navigate to their ''home'' directory with the following command:
 +
[NetID@ada1 ~]$ '''cd /home/''NetID'''''
  
You can navigate to your ''home'' directory with the following command:
+
A User's ''scratch'' directory has more storage space than their ''home'' directory and is recommended for general purpose use. Users can navigate to their ''scratch'' directory with the following command:
  [NetID@ada1 ~]$ '''cd /home/''[NetID]'''''
+
  [NetID@ada1 ~]$ '''cd /scratch/user/''NetID'''''
  
Your ''scratch'' directory has more storage space than your ''home'' directory and is recommended for general purpose use. You can navigate to your ''scratch'' directory with the following command:
+
Users can navigate to ''scratch'' or ''home'' easily by using their respective environment variables.  
[NetID@ada1 ~]$ '''cd /scratch/user/''[NetID]'''''
 
 
 
You can navigate to ''scratch'' or ''home'' easily by using their respective environment variables.  
 
  
 
Navigate to ''scratch'' with the following command:
 
Navigate to ''scratch'' with the following command:
Line 40: Line 56:
  
 
<font color=purple>
 
<font color=purple>
Your ''scratch'' directory is restricted to 1TB/50,000 files of storage. This storage quota is '''expandable''' upon request.
+
A user's ''scratch'' directory is restricted to 1TB/250,000 files of storage. This storage quota is '''expandable''' upon request.  A user's ''scratch directory'' is '''NOT''' backed up.
 
   
 
   
Your ''home'' directory is restricted to 10GB/10,000 files of storage. This storage quota is '''not expandable'''.
+
A user's ''home'' directory is restricted to 10GB/10,000 files of storage. This storage quota is '''not expandable'''.  A user's ''home'' directory is backed up on a nightly basis.
 
</font>
 
</font>
  
You can see the current status of your storage quotas with:
+
Users can see the current status of their storage quotas with:
 
  [NetID@ada1 ~]$ '''showquota'''
 
  [NetID@ada1 ~]$ '''showquota'''
  
Line 55: Line 71:
 
On Ada, '''LSF''' is the batch system that provides job management.  
 
On Ada, '''LSF''' is the batch system that provides job management.  
 
More information on '''LSF''' can be found in the [[Ada:Batch | Ada Batch]] page.
 
More information on '''LSF''' can be found in the [[Ada:Batch | Ada Batch]] page.
 +
== Finding Software ==
 +
 +
Software on Ada is loaded using '''modules'''.
 +
 +
A list of the most popular software on our systems is available on the [[:SW | HPRC Available Software]] page.
 +
 +
To '''find''' ''most'' available software on Ada, use the following command:
 +
[NetID@ada1 ~]$ '''module avail'''
 +
 +
To '''search for''' particular software by keyword, use:
 +
[NetID@ada1 ~]$ '''module spider ''keyword'''''
 +
 +
To load a module, use:
 +
[NetID@ada1 ~]$ '''module load ''moduleName'''''
 +
 +
To list all currently loaded modules, use:
 +
[NetID@ada1 ~]$ '''module list'''
 +
 +
To remove all currently loaded modules, use:
 +
[NetID@ada1 ~]$ '''module purge'''
 +
 +
If you need '''new software''' or '''an update''', please contact us with your request.
 +
 +
There are restrictions on what software we can install. There is also regularly a queue of requested software installations.
  
== Running Your Program / Preparing a Job File ==
+
<font color=teal>Please account for '''delays''' in your installation request timeline. </font>
  
In order to properly run a program on Ada, you will need to create a job file and submit a job.
+
== Running A Program / Preparing a Job File ==
 +
In order to properly run a program on Ada, users will need to create a job file and submit a job.
  
The simple example job file below requests 1 core on 1 node with 2.5GB of RAM for 1.5 hour. Note that most nodes (>800) on Ada have 20 cores with 54GB of usable memory. Some nodes (~26) have 20 cores with 245GB of usable memory. A few special nodes have more cores and memory than this. Please ensure that your job requirements will fit within these restrictions.  
+
The simple example job file below requests 1 core on 1 node with 2.5GB of RAM for 1.5 hour. Note that most nodes (>800) on Ada have 20 cores with 54GB of usable memory. Some nodes (~26) have 20 cores with 245GB of usable memory. A few special nodes have more cores and memory than this. Users are encouraged to ensure that their job requirements will fit within these restrictions.  
  
 
Any modules that need to be loaded or executable commands will replace the ''"#First Executable Line"'' in this example.
 
Any modules that need to be loaded or executable commands will replace the ''"#First Executable Line"'' in this example.
Line 78: Line 119:
 
Note: If your job file has been written on an older Mac or DOS workstation, you will need to use "dos2unix" to remove certain characters that interfere with parsing the script.
 
Note: If your job file has been written on an older Mac or DOS workstation, you will need to use "dos2unix" to remove certain characters that interfere with parsing the script.
  
  [NetID@terra1 ~]$ '''dos2unix ''MyJob.slurm'''''
+
  [NetID@ada1 ~]$ '''dos2unix ''MyJob.lsf'''''
  
More information on '''job options''' can be found in the [[Terra:Batch#Building_Job_Files | Building Job Files]] section of the [[Terra:Batch | Terra Batch]] page.
+
More information on '''job options''' can be found in the [[Ada:Batch#Building_Job_Files | Building Job Files]] section of the [[Ada:Batch | Ada Batch]] page.
  
 
More information on '''dos2unix''' can be found on the [[:SW:dos2unix | dos2unix]] section of the [[:SW | HPRC Available Software]] page.
 
More information on '''dos2unix''' can be found on the [[:SW:dos2unix | dos2unix]] section of the [[:SW | HPRC Available Software]] page.
  
 
== Submitting and Monitoring Jobs ==
 
== Submitting and Monitoring Jobs ==
Once you have your job file ready, it is time to submit your job. You can submit your job to slurm with the following command:
+
Once a job file is ready, it is time to submit the job. Users can submit their jobs to LSF with the following command:
  [NetID@terra1 ~]$ '''sbatch ''MyJob.slurm'''''
+
  [NetID@ada1 ~]$ '''bsub < ''MyJob.lsf'''''
  Submitted batch job 3606
+
  Verifying job submission parameters...
 +
Verifying project account...
 +
      Account to charge:  000000000000
 +
          Balance (SUs):      5000.0000
 +
          SUs to charge:        1.5000
 +
Job <0000000> is submitted to default queue <sn_short>.
  
After the job has been submitted, you are able to monitor it with several methods.  
+
After the job has been submitted, users are able to monitor their own jobs with several methods.  
 
To see the status of all of your jobs, use the following command:
 
To see the status of all of your jobs, use the following command:
  [NetID@terra1 ~]$ '''squeue -u ''NetID'''''
+
  [NetID@ada1 ~]$ '''bjobs'''
  JOBID       NAME                USER                   PARTITION  NODES CPUS STATE      TIME       TIME_LEFT  START_TIME          REASON      NODELIST            
+
  JOBID     STAT  USER             QUEUE      JOB_NAME            NEXEC_HOST SLOTS RUN_TIME       TIME_LEFT
  3606        myjob2              NetID                   short      1    3    RUNNING     0:30        00:10:30   2016-11-27T23:44:12  None        tnxt-[0340]  
+
0000000    RUN   NetID           sn_short  JobExample1          1          1    0 second(s)    1:30 L 
 +
  0000001    PEND  NetID           sn_short  JobExample1          1          1    0 second(s)     1:30 L  
  
To see the status of one job, use the following command, where ''XXXX'' is the JobID:
+
To see the status of one job, use the following command, where ''XXXXXXX'' is the JobID:
  [NetID@terra1 ~]$ '''squeue --job ''XXXX'''''
+
  [NetID@ada1 ~]$ '''bjobs ''XXXXXXX'''''
  JOBID       NAME                USER                   PARTITION  NODES CPUS STATE      TIME       TIME_LEFT   START_TIME          REASON      NODELIST           
+
  JOBID     STAT  USER             QUEUE      JOB_NAME            NEXEC_HOST SLOTS RUN_TIME       TIME_LEFT
  XXXX        myjob2              NetID                   short      1    3    RUNNING     0:30        00:10:30   2016-11-27T23:44:12  None        tnxt-[0340]  
+
  XXXXXXX    RUN  NetID           sn_short  JobExample1          1          1    0 second(s)     1:30 L  
  
To cancel a job, use the following command, where ''XXXX'' is the JobID:
+
To cancel a job, use the following command, where ''XXXXXXX'' is the JobID:
  [NetID@terra1 ~]$ '''scancel ''XXXX'''''
+
  [NetID@ada1 ~]$ '''bkill ''XXXXXXX'''''
 +
Job <XXXXXXX> is being terminated
  
More information on '''submitting''' and '''monitoring''' Slurm jobs can be found in the [[:Terra:Batch#Job_Submission | Job Submission]] section of the [[:Terra:Batch | Terra Batch System]] page.
+
More information on '''submitting''' and '''monitoring''' LSF jobs can be found in the [[:Ada:Batch#Job_Submission | Job Submission]] section of the [[:Ada:Batch | Ada Batch System]] page.
  
 
== Additional Topics ==
 
== Additional Topics ==
Line 112: Line 160:
  
 
Our staff has also written some example jobs for specific software. These software-specific examples can be seen on the [[:SW | Individual Software Pages]] where available.  
 
Our staff has also written some example jobs for specific software. These software-specific examples can be seen on the [[:SW | Individual Software Pages]] where available.  
 
=== Finding Software ===
 
 
Software on Terra is loaded using '''modules'''.
 
 
You can see the most popular software on the [[:SW | HPRC Available Software]] page.
 
 
You can '''find''' ''most'' available software on Terra with the following command:
 
[NetID@terra1 ~]$ '''module avail'''
 
 
You can '''search for''' particular software by keyword using:
 
[NetID@terra1 ~]$ '''module spider ''keyword'''''
 
 
You can load a module using:
 
[NetID@terra1 ~]$ '''module load ''moduleName'''''
 
 
You can list all currently loaded modules using:
 
[NetID@terra1 ~]$ '''module list'''
 
 
You can remove all currently loaded modules using:
 
[NetID@terra1 ~]$ '''module purge'''
 
 
If you need '''new software''' or '''an update''', please contact us with your request.
 
 
There are restrictions on what software we can install. There is also regularly a queue of requested software installations.
 
 
<font color=teal>Please account for '''delays''' in your installation request timeline. </font>
 
  
 
=== Transferring Files ===
 
=== Transferring Files ===
  
Files can be transferred to Terra using the ''scp'' command or a file transfer program.
+
Files can be transferred to Ada using the ''scp'' command or a file transfer program.
  
 
Our users most commonly utilize:
 
Our users most commonly utilize:
 
* [https://winscp.net/eng/download.php WinSCP] - Straightforward, legacy
 
* [https://winscp.net/eng/download.php WinSCP] - Straightforward, legacy
* [https://filezilla-project.org/ FileZilla Client] - Easy to use, additional features, available on most platforms
+
<!--* [https://filezilla-project.org/ FileZilla Client] - Easy to use, additional features, available on most platforms-->
 
* [https://mobaxterm.mobatek.net/features.html MobaXterm Graphical SFTP] - Included with MobaXterm
 
* [https://mobaxterm.mobatek.net/features.html MobaXterm Graphical SFTP] - Included with MobaXterm
  
See our [Terra-Filezilla example video] for a demonstration of this process.
+
<!-- See our [Ada-Filezilla example video] for a demonstration of this process. -->
 
+
<font color=teal>'''Advice:''' while GUIs are acceptable for file transfers, the cp and scp commands are much quicker and may significantly benefit workflow.</font>
<font color=teal>'''Advice:''' while GUIs are acceptable for file transfers, the cp and scp commands are much quicker and may significantly benefit your workflow.</font>
 
  
 
==== Reliably Transferring Large Files ====
 
==== Reliably Transferring Large Files ====
  
For files larger than several GB, you will want to consider the use of a more fault-tolerant utility such as rsync.
+
For files larger than several GB, users will want to consider the use of a more fault-tolerant utility such as rsync.
  [NetID@terra1 ~]$ '''rsync -av [-z] ''localdir/ userid@remotesystem:/path/to/remotedir/'''''
+
  [NetID@ada1 ~]$ '''rsync -av [-z] ''localdir/ userid@remotesystem:/path/to/remotedir/'''''
  
 
An rsync example can be seen on the [[:Ada:Fast_Data_Transfer#Data_transfer_using_rsync | Ada Fast Transfer]] page.
 
An rsync example can be seen on the [[:Ada:Fast_Data_Transfer#Data_transfer_using_rsync | Ada Fast Transfer]] page.
 
+
<!-- See our [Ada-rsync example video] for a demonstration of this process.-->
See our [Terra-rsync example video] for a demonstration of this process.
+
<!-- [Insert info on glob, ftn] -->
 
 
[Insert info on glob, ftn]
 
  
 
=== Graphic User Interfaces (Visualization) ===
 
=== Graphic User Interfaces (Visualization) ===
  
The use of GUIs on Terra is a more complicated process than running non-interactive jobs or doing resource-light interactive processing.
+
The Open on Demand Portal is the preferred way to access visual applications on Ada. From the [https://portal.hprc.tamu.edu OOD portal], you can browse and manage the files in your ''home'' and ''scratch'' directories. This allows user's to easily view media files (.png .jpeg .mp4 etc) through their web browser of choice.  
  
You have '''two options''' for using GUIs on Terra.
+
Here's a quick summary of the functions of the OOD Portal:
  
The '''first option''' is to run on the login node. When doing this, you '''must''' observe the fair-use policy of login node usage. Users commonly violate these policies by accident, resulting in terminated processes, confusion, and warnings from our admins.
+
*View and submit visualization jobs
 +
*View and submit interactive applications such as Matlab & ANSYS
 +
*View your active jobs on Ada
 +
*Access the shell of Ada, Terra, and Curie
 +
*View and manage the files in your Ada home and scratch directories
 +
<!--The documentation from the viz-portal short course including usage instructions can be found on the respective [https://hprc.tamu.edu/training/intro_vis_portal.html short course page]. -->
 +
<!--The VNC method is outside the scope of this guide. See the [[Ada:Remote-Viz | Ada Remote Visualization]] page for more information.-->
  
The '''second option''' is to use a VNC job. This method is outside the scope of this guide. See the [[Terra:Remote-Viz | Terra Remote Visualization]] page for more information.
+
=== Requesting Shared Space on Ada ===
 +
Shared spaces are directories that can be accessed by multiple authorized users.  In order to request a shared space, users must submit their request to [mailto:help@hprc.tamu.edu help@hprc.tamu.edu] with the following information:
 +
* Number of files to be stored in the shared directory (approximate)
 +
* Total amount of data to be stored in the shared directory (approximate)
 +
* Primary contact for authorizing access to the shared directory
 +
* Initial users to have access to the shared directory after creation
 +
The location of the shared directory will be in <pre>/home/[NetID]/scratch/group</pre>
  
[[Category: Terra]]
+
[[Category: Ada]]

Revision as of 15:59, 2 June 2020

Ada Quick Start Guide

Ada Usage Policies

Access to Ada is granted with the condition that you will understand and adhere to all TAMU HPRC and Ada-specific policies.

General policies can be found on the HPRC Policies page.

Ada-specific policies can be found on the Ada Policies page.

Accessing Ada

Most access to Ada is done via a secure shell session. In addition, two-factor authentication is required to login to any cluster.

Users on Windows computers use either PuTTY or MobaXterm. If MobaXterm works on your computer, it is usually easier to use. When starting an ssh session in PuTTY, choose the connection type 'SSH', select port 22, and then type the hostname 'ada.tamu.edu'. For MobaXterm, select 'Session', 'SSH', and then remote host 'ada.tamu.edu'. Check the box to specify username and type your NetID. After selecting 'Ok', you will be prompted for Duo Two Factor Authentication. For more detailed instructions, visit the Two Factor Authentication page.

Users on Mac and Linux/Unix should use whatever SSH-capable terminal is available on their system. The command to connect to Ada is as follows. Be sure to replace [NetID] with your TAMU NetID.

[user1@localhost ~]$ ssh [NetID]@ada.tamu.edu

Note: In this example [user1@localhost ~]$ represents the command prompt on your local machine.
Your login password is the same that used on Howdy. You will not see your password as your type it into the login prompt.

Off Campus Access

Please visit this page to find information on accessing Ada remotely.

For more detailed instructions on how to access our systems, please see the HPRC Access page.

Navigating Ada & Storage Quotas

When users first access Ada, they will be within their home directory. This directory has smaller storage quotas and should not be used for general purpose.

Users can navigate to their home directory with the following command:

[NetID@ada1 ~]$ cd /home/NetID

A User's scratch directory has more storage space than their home directory and is recommended for general purpose use. Users can navigate to their scratch directory with the following command:

[NetID@ada1 ~]$ cd /scratch/user/NetID

Users can navigate to scratch or home easily by using their respective environment variables.

Navigate to scratch with the following command:

[NetID@ada1 ~]$ cd $SCRATCH

Navigate to home with the following command:

[NetID@ada1 ~]$ cd $HOME

A user's scratch directory is restricted to 1TB/250,000 files of storage. This storage quota is expandable upon request. A user's scratch directory is NOT backed up.

A user's home directory is restricted to 10GB/10,000 files of storage. This storage quota is not expandable. A user's home directory is backed up on a nightly basis.

Users can see the current status of their storage quotas with:

[NetID@ada1 ~]$ showquota

If you need a storage quota increase, please contact us with justification and the expected length of time that you will need the quota increase.

The Batch System

The batch system is a load distribution implementation that ensures convenient and fair use of a shared resource. Submitting jobs to a batch system allows a user to reserve specific resources with minimal interference to other users. All users are required to submit resource-intensive processing to the compute nodes through the batch system - attempting to circumvent the batch system is not allowed.

On Ada, LSF is the batch system that provides job management. More information on LSF can be found in the Ada Batch page.

Finding Software

Software on Ada is loaded using modules.

A list of the most popular software on our systems is available on the HPRC Available Software page.

To find most available software on Ada, use the following command:

[NetID@ada1 ~]$ module avail

To search for particular software by keyword, use:

[NetID@ada1 ~]$ module spider keyword

To load a module, use:

[NetID@ada1 ~]$ module load moduleName

To list all currently loaded modules, use:

[NetID@ada1 ~]$ module list

To remove all currently loaded modules, use:

[NetID@ada1 ~]$ module purge

If you need new software or an update, please contact us with your request.

There are restrictions on what software we can install. There is also regularly a queue of requested software installations.

Please account for delays in your installation request timeline.

Running A Program / Preparing a Job File

In order to properly run a program on Ada, users will need to create a job file and submit a job.

The simple example job file below requests 1 core on 1 node with 2.5GB of RAM for 1.5 hour. Note that most nodes (>800) on Ada have 20 cores with 54GB of usable memory. Some nodes (~26) have 20 cores with 245GB of usable memory. A few special nodes have more cores and memory than this. Users are encouraged to ensure that their job requirements will fit within these restrictions.

Any modules that need to be loaded or executable commands will replace the "#First Executable Line" in this example.

##NECESSARY JOB SPECIFICATIONS
#BSUB -J JobExample1         #Set the job name to "JobExample1"
#BSUB -L /bin/bash           #Uses the bash login shell to initialize the job's execution environment.
#BSUB -W 1:30                #Set the wall clock limit to 1hr and 30min
#BSUB -n 1                   #Request 1 task
#BSUB -R "span[ptile=1]"     #Request 1 task/core per node
#BSUB -R "rusage[mem=2560]"  #Request 2560MB (2.5GB) per node
#BSUB -M 2560                #Set the per process enforceable memory limit to 2560MB
#BSUB -o Example1Out.%J      #Send stdout/err to "Example1Out.[jobID]"

#First Executable Line

Note: If your job file has been written on an older Mac or DOS workstation, you will need to use "dos2unix" to remove certain characters that interfere with parsing the script.

[NetID@ada1 ~]$ dos2unix MyJob.lsf

More information on job options can be found in the Building Job Files section of the Ada Batch page.

More information on dos2unix can be found on the dos2unix section of the HPRC Available Software page.

Submitting and Monitoring Jobs

Once a job file is ready, it is time to submit the job. Users can submit their jobs to LSF with the following command:

[NetID@ada1 ~]$ bsub < MyJob.lsf
Verifying job submission parameters...
Verifying project account...
     Account to charge:   000000000000
         Balance (SUs):      5000.0000
         SUs to charge:         1.5000
Job <0000000> is submitted to default queue <sn_short>.

After the job has been submitted, users are able to monitor their own jobs with several methods. To see the status of all of your jobs, use the following command:

[NetID@ada1 ~]$ bjobs
JOBID      STAT  USER             QUEUE      JOB_NAME             NEXEC_HOST SLOTS RUN_TIME        TIME_LEFT  
0000000    RUN   NetID            sn_short   JobExample1          1          1     0 second(s)     1:30 L  
0000001    PEND  NetID            sn_short   JobExample1          1          1     0 second(s)     1:30 L  

To see the status of one job, use the following command, where XXXXXXX is the JobID:

[NetID@ada1 ~]$ bjobs XXXXXXX
JOBID      STAT  USER             QUEUE      JOB_NAME             NEXEC_HOST SLOTS RUN_TIME        TIME_LEFT  
XXXXXXX    RUN   NetID            sn_short   JobExample1          1          1     0 second(s)     1:30 L  

To cancel a job, use the following command, where XXXXXXX is the JobID:

[NetID@ada1 ~]$ bkill XXXXXXX
Job <XXXXXXX> is being terminated

More information on submitting and monitoring LSF jobs can be found in the Job Submission section of the Ada Batch System page.

Additional Topics

Translating Ada/LSF <--> Terra/Slurm

The HPRC Batch Translation page contains information on converting between LSF, PBS, and Slurm.

Our staff has also written some example jobs for specific software. These software-specific examples can be seen on the Individual Software Pages where available.

Transferring Files

Files can be transferred to Ada using the scp command or a file transfer program.

Our users most commonly utilize:

Advice: while GUIs are acceptable for file transfers, the cp and scp commands are much quicker and may significantly benefit workflow.

Reliably Transferring Large Files

For files larger than several GB, users will want to consider the use of a more fault-tolerant utility such as rsync.

[NetID@ada1 ~]$ rsync -av [-z] localdir/ userid@remotesystem:/path/to/remotedir/

An rsync example can be seen on the Ada Fast Transfer page.

Graphic User Interfaces (Visualization)

The Open on Demand Portal is the preferred way to access visual applications on Ada. From the OOD portal, you can browse and manage the files in your home and scratch directories. This allows user's to easily view media files (.png .jpeg .mp4 etc) through their web browser of choice.

Here's a quick summary of the functions of the OOD Portal:

  • View and submit visualization jobs
  • View and submit interactive applications such as Matlab & ANSYS
  • View your active jobs on Ada
  • Access the shell of Ada, Terra, and Curie
  • View and manage the files in your Ada home and scratch directories

Requesting Shared Space on Ada

Shared spaces are directories that can be accessed by multiple authorized users. In order to request a shared space, users must submit their request to help@hprc.tamu.edu with the following information:

  • Number of files to be stored in the shared directory (approximate)
  • Total amount of data to be stored in the shared directory (approximate)
  • Primary contact for authorizing access to the shared directory
  • Initial users to have access to the shared directory after creation
The location of the shared directory will be in
/home/[NetID]/scratch/group