Hprc banner tamu.png

SW:Installing New Software

From TAMU HPRC
Jump to: navigation, search

Installing New Software

Introduction

A comprehensive list of available software modules and how to find them can be found here. If you find that particular package is not supported by the systems, it is possible to install software in your $SCRATCH directory. Software installations by HPRC staff are usually reserved for generalized/popular or complex installations. There is a 10-15 business day (2-3 week) turn-around time for software installation requests.

You are encouraged to save time by trying to install new software via the process in the following sections.


General Installation Notes

User installation of packages is straightforward except for a few conditions. The following installation notes comprise most issues users encounter.

  • Disk/File Quota: Storage within $HOME is limited. Packages should be installed in user's $SCRATCH. See Ada File Systems or Terra File Systems for more info.
  • Internet Connection: Only login nodes have access to the Internet. Attempting to use pip or other Internet access from a batch job or a portal app, both of which run on a compute node, will fail.
  • Mixing Tools: myPython, Anaconda, and myAnaconda must be used exclusively of one another. Attempting to mix these in order to add a package/library will fail.
  • Version Compatibility: Some packages require specific/older versions of software to be installed. Please verify package compatibility against available versions.

Installing software with No Dependencies

If a software has no dependencies, it means it requires no software other than the operating system to function. This type of software is the simplest to install. One should always check to see if any dependencies exist for the software before attempting to install it.

Example: Installing readline from source

Readline is a software available on the GNU website. The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are typed in.

To download the source:

In many cases such as this, the download will be a compressed file which must be unpacked according to its extension.

[NetID@ada]$ cd $SCRATCH
[NetID@ada]$ wget ftp://ftp.gnu.org/gnu/readline/readline-6.3.tar.gz

To unpack a tar.gz file use:

This will unpack the contents of the compressed file into a directory of the same name.

[NetID@ada]$ tar -xzf readline-6.3.tar.gz

Different options are available for the tar command that will allow the unpacking of other compressed file types.
To learn more:

[NetID@ada]$ man tar

To get into the unpacked directory:

In most cases, the contents will include a file called INSTALL README, which will contain package-specific information on the installation process. This information can be viewed at any time by opening the INSTALL or README file in any text editor.

[NetID@ada]$ cd readline-6.3

To configure readline in a separate directory:

If a path is not specified, make install will attempt to write to /usr/local which is off limits to users.

[NetID@ada]$ ./configure --prefix =/scratch/user/[NetID]/

To compile readline:

[NetID@ada]$ make 
[NetID@ada]$ make install

Readline is now available for use on your shell. Its commands are carried out with key shortcuts such as ctrl - A to move to the start of a line or ctrl - E to move to the end of the line.


Installing Python Packages

While multiple versions of Python, Anaconda, and TensorFlow are available on our clusters, it is at times desired to have some specialized libraries or packages installed in addition to our pre-installed software. Pay careful attention to which versions of Python are supported by your package, and be sure to choose the corresponding module.

For a more detailed walkthrough of user installation using virtual environments, see our Python page.

To install a new Python Package:

[NetID@terra ~]$ ml purge
[NetID@terra ~]$ ml Python/3.5.2-intel-2017A
[NetID@terra ~]$ virtualenv tdenv
[NetID@terra ~]$ source tdenv/bin/activate
[NetID@terra ~]$ pip install [link to package file]

Installing with Conda

It is recommended to install new packages in a private virtual environment. For more detailed information on virtual environments, please see Anaconda page.

Create a Private Anaconda Virtual Environment

Make scratch directory as your current directory and follow the commands in order to create your own virtual environment. NOTE: Do not create an environment in your $HOME directory (/home/yourNetID). You will exceed your $HOME directory file limit. Instead, please install into your $SCRATCH directory (/scratch/user/yourNetID).

   [NetID@cluster NetID]$ cd $SCRATCH                          # Make scratch your current directory
   [NetID@cluster NetID]$ module load Anaconda/3-5.0.0.1       # Load Anaconda module
   [NetID@cluster NetID]$ conda create --name myenv            # Create environment 

Now "conda info --env" command will also show your private environment.

Activate a virutal environment

    [NetID@cluster NetID]$ cd $SCRATCH 
    [NetID@cluster NetID]$ module load Anaconda/3-5.0.0.1          # Load Anaconda module
    [NetID@cluster NetID]$ source activate myenv                   # Activate environment 

Install/Uninstall Packages in a Anaconda Virtual Environment

NOTE: Users can only install/uninstall packages in their private environment. Users don't have access to install/uninstall packages in root and shared environments.

To install/uninstall packages in private environments users first need to activate them. For example, next few steps show how to install and uninstall numpy package in the "myenv" private environment.

  [NetID@cluster NetID]$ cd $SCRATCH 
  [NetID@cluster NetID]$ module load Anaconda/3-5.0.0.1          # Load Anaconda module
  [NetID@cluster NetID]$ source activate myenv                   # Activate environment
  (myenv) [NetID@cluster NetID]$ conda install numpy             # Command to install numpy package
  (myenv) [NetID@cluster NetID]$ conda list                      # Conda list command to check packages
  (myenv) [NetID@cluster NetID]$ conda uninstall numpy           # Command to uninstall numpy package