Hprc banner tamu.png

Difference between revisions of "Terra:QuickStart"

From TAMU HPRC
Jump to: navigation, search
(Finding Software)
(Additional Topics)
Line 87: Line 87:
 
== Additional Topics ==
 
== Additional Topics ==
  
=== Finding Output ===
+
=== Translating Ada/LSF <--> Terra/Slurm ===
 +
 
 +
The [[:HPRC:Batch_Translation | HPRC Batch Translation]] page contains information on '''converting''' between LSF, PBS, and Slurm.
 +
 
 +
Our staff has also written some example jobs for specific software. These software-specific examples can be seen on the [[:SW | Individual Software Pages]] where available.
  
 
=== Finding Software ===
 
=== Finding Software ===
Line 108: Line 112:
  
 
=== Transferring Files ===
 
=== Transferring Files ===
 +
 +
Files can be transferred to Terra using the ''scp'' command or a file transfer program.
 +
 +
Our users most commonly utilize:
 +
[[https://winscp.net/eng/download.php WinSCP]] - Straightforward, legacy
 +
[[https://filezilla-project.org/ FileZilla Client]] - Easy to use, additional features, available on most platforms
 +
[[https://mobaxterm.mobatek.net/features.html MobaXterm Graphical SFTP]] - Included with MobaXterm
 +
 +
See our [Terra-Filezilla example video] for a demonstration of this process.
 +
 +
<font color=teal>'''Advice:''' while GUIs are acceptable for file transfers, the cp and scp commands are much quick and may significantly benefit your workflow.</font>
 +
 +
[Insert info on glob, rsync, ftn]
  
 
=== Graphic User Interfaces (Visualization) ===
 
=== Graphic User Interfaces (Visualization) ===
Line 118: Line 135:
  
 
The '''second option''' is to use a VNC job. This method is outside the scope of this guide. See the [[Terra:Remote-Viz | Terra Remote Visualization]] page for more information.
 
The '''second option''' is to use a VNC job. This method is outside the scope of this guide. See the [[Terra:Remote-Viz | Terra Remote Visualization]] page for more information.
 
=== Translating Ada/LSF <--> Terra/Slurm ===
 
 
The [[:HPRC:Batch_Translation | HPRC Batch Translation]] page contains information on '''converting''' between LSF, PBS, and Slurm.
 
 
Some LSF-Slurm translation examples have also been made available on the ADA-TERRA JOBS TRANSLATION PAGE.
 
Our staff has also written some example jobs for specific software. These software-specific examples can be seen on the INDIVIDUAL SOFTWARE PAGES where available.
 
 
  
 
[[Category: Terra]]
 
[[Category: Terra]]

Revision as of 13:38, 18 November 2016

Terra Quick Start Guide

Terra Usage Policies

Access to Terra is granted with the condition that you will understand and adhere to all TAMU HPRC and Terra-specific policies.

General policies can be found on the HPRC Policies page.

Terra-specific policies, which are similar to Ada, can be found on the Terra Policies page.

Accessing Terra

Most access to Terra is done via a secure shell session.

Users on Windows computers use either PuTTY or MobaXterm. If MobaXterm works on your computer, it is usually easier to use.

Users on Mac and Linux/Unix should use whatever SSH-capable terminal is available on their system.

The command to connect to Terra is as follows. Be sure to replace [NetID] with your TAMU NetID.

ssh [NetID]@terra.tamu.edu

Your login password is the same that used on Howdy. You will not see your password as your type it into the login prompt.

Navigating Terra & Storage Quotas

When you first access Terra, you will be within your home directory. This directory has smaller quotas and should not be used for general purpose.

The home directory is located at:

/home/[NetID]

You will want to navigate to your scratch directory. This directory is located at:

/scratch/user/[NetID]

You can navigate to scratch easily with the following command:

cd $SCRATCH

Your scratch directory is restricted to 1TB/50,000 files of storage. This quota is extendable upon request.

Your home directory is restricted to 10GB/10,000 files of storage. This quota is not extendable.

You can see the current status of your quota with:

showquota

If you need a quota increase, please contact us with justification and the expected length of time that you will need the extended quota.

The Batch System

The batch system is a load distribution implementation that ensures convenient and fair use of a shared resource. Submitting jobs to a batch system allows a user to reserve specific resources with minimal interference to other users. All users are required to submit resource-intensive processing to the compute nodes through the batch system - attempting to circumvent the batch system is not allowed.

On Terra, Slurm is the batch system that provides job management. More information on Slurm can be found in the Terra Batch page.

Running Your Program / Preparing a Job File

In order to properly run a program on Terra, you will need to submit a job.

The simple example below request 1 core on 1 node with 5GB of RAM for 1 hour. "ExampleModule" should be replaced or omitted based on your software needs.

#!/bin/bash
#SBATCH --export=NONE
#SBATCH --get-user-env=L
#SBATCH -t 01:00:00
#SBATCH -J ExampleJob
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --mem=5G
#SBATCH -o ExJobOut.%j
module load ExampleModule
./myprogram

This job file should be written on a Linux/Unix environment.

If written on an older Mac or DOS workstation, you will need to use "dos2unix" to remove certain characters that interfere with parsing the script.

dos2unix myExJobFile.slurm

More information on job options can be found in the Building Job Files section of the Terra Batch page.

More information on dos2unix can be found on the dos2unix section of the HPRC Available Software page.

Submitting and Monitoring Jobs

After submitted a job, you are able to monitor it with several methods.


More information on submitting and monitoring Slurm jobs can be found in the Job Submission section of the Terra Batch System page.

Additional Topics

Translating Ada/LSF <--> Terra/Slurm

The HPRC Batch Translation page contains information on converting between LSF, PBS, and Slurm.

Our staff has also written some example jobs for specific software. These software-specific examples can be seen on the Individual Software Pages where available.

Finding Software

Software on Terra is loaded using modules.

You can see the most popular software on the HPRC Available Software page.

You can find most available software on Terra with the following command:

module avail

You can search for particular software by keyword using:

module spider keyword

If you need new software or an update, please contact us with your request.

There are restrictions on what software we can install. There is also regularly a queue of requested software installations.

Account for delays in your installation request timeline.

Transferring Files

Files can be transferred to Terra using the scp command or a file transfer program.

Our users most commonly utilize: [WinSCP] - Straightforward, legacy [FileZilla Client] - Easy to use, additional features, available on most platforms [MobaXterm Graphical SFTP] - Included with MobaXterm

See our [Terra-Filezilla example video] for a demonstration of this process.

Advice: while GUIs are acceptable for file transfers, the cp and scp commands are much quick and may significantly benefit your workflow.

[Insert info on glob, rsync, ftn]

Graphic User Interfaces (Visualization)

The use of GUIs on Terra is a more complicated process than running non-interactive jobs or doing resource-light interactive processing.

You have two options for using GUIs on Terra.

The first option is to run on the login node. When doing this, you must observe the fair-use policy of login node usage. Users commonly violate these policies by accident, resulting in terminated processes, confusion, and warnings from our admins.

The second option is to use a VNC job. This method is outside the scope of this guide. See the Terra Remote Visualization page for more information.