Skip to content

ModuLair

Description

ModuLair is a Python virtual environment (venv) management solution developed here at HPRC. It empowers users to streamline their Python workflows without having to spend time configuring third-party solutions to integrate with HPC. ModuLair is pre-installed on our clusters and provides seamless integration with our systems.

Command Line Interface

ModuLair's Command Line Interface offers four commands that compose the backbone of the software's utility.

  1. create_venv - Create a new virtual environment in any available Python version. Add a description of what the environment is for and/or a custom module toolchain tailored to your workflow.

  2. list_venvs - List all of the environments that you have created with the ModuLair suite. This is your "Lair."

  3. activate_venv - Activate any of the environments in your Lair with one command.

  4. delete_venv - Delete all of the data of any of the environments in your Lair.

Example environment life-cycle

This example shows the life-cycle of a venv for a deep learning project using gpu accelration. This environment will be created with a custom toolchain by leveraging the -t flag. This option accepts a space-separated list of modules as an input. 

$ create_venv dl_train -d "Environment for training my deep learning model" -t "CUDA/12.8.0 GCCcore/13.3.0 Python/3.12.3"
NOTE: When providing a custom toolchain, make sure dependencies of a module are written before it. For example, if Python was provided before GCCcore, this command would fail, because the Python module relies on the GCCcore module. 
$ list_venvs
Name                Description              Python Version    GCC Version    Toolchain
------------------------------------------------------------------------------------------------------------
dl_train            Environment for training my deep learning model    N/A               N/A            CUDA/12.8.0 GCCcore/13.3.0 Python/3.12.3

$ source activate_venv dl_train
(dl_train) $ ... your commands

$ delete_venv dl_train
Removing {'name': 'dl_train', 'python_version': 'N/A', 'GCCcore_version': 'N/A', 'description': 'Environment for training my deep learning model', 'toolchain': 'CUDA/12.8.0 GCCcore/13.3.0 Python/3.12.3'}'s files...

Successfully removed {'name': 'dl_train', 'python_version': 'N/A', 'GCCcore_version': 'N/A', 'description': 'Environment for training my deep learning model', 'toolchain': 'CUDA/12.8.0 GCCcore/13.3.0 Python/3.12.3'}

OnDemand Portal

Each of HPRC's clusters has an OnDemand Portal which contains many tools to manage a user's HPRC resources. ModuLair's graphical interface can be found as an available tool here.

The GUI offers the create, delete, and list functionality provided by the CLI. The GUI makes it easier for users that are unfamiliar with the command line to set up their workflows.