You can create your own personal conda environment using Anaconda which installs conda and bioconda packages.
You can install software that supports conda/bioconda installation inside the conda environment you create.
Once you create a conda environment, you can continue to install other packages into that environment.
You can create more than one environment in case you need specific versions of tools.
Before you begin, you will need to contact the HPRC helpdesk (firstname.lastname@example.org) to request a $SCRATCH file quota increase because creating a conda environment will add thousands of files to your $SCRATCH space.
After you get your file quota increase, the next thing you need to do is to add a few conda channels which contain certain bioinformatics tools.
You only need to do this step once in order to add the conda-forge and bioconda channels:
module purge module load Anaconda/3-184.108.40.206 conda config --add channels defaults conda config --add channels conda-forge conda config --add channels bioconda
Create a new conda environment
To to create an Anaconda conda environment called bio, do the following on the command line:
module purge module load Anaconda/3-220.127.116.11 conda create -n bio
After your bio environment is created, you will see output on how to activate and use your bio environment
# # To activate this environment, use: # > source activate bio # # To deactivate an active environment, use: # > source deactivate #
Then you can add packages to your bio environment
source activate bio conda install -c bioconda bwa
To to create an Miniconda conda environment called jupyterlab_1.2.2, do the following on the command line:
module purge module load Miniconda3/4.7.10 conda create -p /scratch/user/your_net_id/.conda/envs/jupyterlab_1.2.2 jupyterlab=1.2.2
After your jupyterlab_1.2.2 environment is created, you will see output on how to activate and use your bio environment
# # To activate this environment, use # # $ conda activate /scratch/user/netid/.conda/envs/jupyterlab_1.2.2 # # To deactivate an active environment, use # # $ conda deactivate
You can add packages to your Miniconda3 environment using either Anaconda/3-18.104.22.168 or Miniconda3/4.7.10 both which use python v3.6.7
When activating the conda environment using the Miniconda3 module, you must specify the full path. When using the Anaconda module you only need to specify the environment name.
In this example, JupyterLab should be run using the portal JupyterLab app. You can use your Miniconda3/4.7.10 environment in the JupyterLab portal app by selecting the Anaconda/3-22.214.171.124 module in the portal app page and providing the name of your Miniconda3/4.7.10 environment in the "JupyterLab Environment to be activated" box.
Install conda/bioconda packages into an existing environment
If you want to install packages such as fastq-multx and bwa into your bio environment (notice the (bio) indicating you are within the bio environment):
[net_id@ada7 any_directory]$ module purge [net_id@ada7 any_directory]$ module load Anaconda/3-126.96.36.199 [net_id@ada7 any_directory]$ source activate bio (bio) [net_id@ada7 any_directory]$ conda install fastq-multx bwa
Debug Failed Installation
Although Anaconda is configured to install in the user's $SCRATCH/.conda directory, some anaconda packages may utilize the $HOME/.local directory during installation which could cause you to reach your file quota for your $HOME directory. To resolve this issue, move your ~/.local directory to your $SCRATCH directory and create a symbolic link in your home directory to the .local scratch directory:
cd mv .local $SCRATCH ln -s $SCRATCH/.local
Using your conda Environment
To use your bio environment in a job script, you must first activate the bio environment and then run tool commands (fastq-multx in this example). Here is a sample job script (excluding the #BSUB headers)
module load Anaconda/3-188.8.131.52 source activate bio fastq-multx -l barcodes.grp seq2.fastq.gz seq1.fastq.gz -o n/a -o out%.fq source deactivate