SW:EasyBuild

EasyBuild

Introduction to EasyBuild

"EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way."

It is the primary tool we use for deploying sofware on HPRC clusters. Benefits include:

1. allows us to deploy software to the GPFS shared filesystem instead of via RPMs which:
   1. cuts down on the compute node image size which:
      1. cuts down on deployment time when nodes are updated
      2. minimizes RPMs, and thus the overall disk image size, for compute nodes that use RAMdisk for local storage (the larger the disk image the less RAM that is available for computation)
      3. allows us to provide all nodes with updates with no further effort
2. allows us to deploy multiple versions of software built with multiple toolchains
3. provides automatic logging and build statistics for every software installation
4. provides automatic module creation
5. provides exact dependency tracking
6. supports common software installation tasks like using configure/make/make install, cmake/make or python -c "setup.py build"

HPRC is actively involved with EasyBuild. In fact, the codes that support IBM Power were developed here on the curie cluster. A number of the modules for the 2017b toolchain also came from here. In addition, we do regular testing and offer new modules (as time permits).

For complete documentation on using EasyBuild, see their readthedocs.io site. After loading the EasyBuild-<clustername>-SCRATCH module below you should immediately be able to start using EasyBuild using the documentation at that site.

To track EasyBuild development, see the EasyBuild GitHub site. Most users will be interesting in the "easyconfigs" repository to see different examples of using EasyBuild.

Using EasyBuild to build software on HPRC clusters

The core EasyBuild module provides the basis for using EasyBuild. This is installed via EasyBuild and is regularly updated when new versions are available.

For years, HPRC admins have been using an EasyBuild-<cluster> module (.e.g EasyBuild-ada) that sets the needed EASYBUILD environment variables for deploying software on our cluster. By default, this uses the latest version of EasyBuild installed.

As of December 2018, we now have an EasyBuild-<cluster>-SCRATCH that loads the above modules and then sets things up so users can use EasyBuild to deploy software to their $SCRATCH/eb directory.

Using your EasyBuild built software

In order to use your software built with EasyBuild, there is no need to load the entire EasyBuild environment (in fact in most case you won't want to). We provide the 'myEB' module that adds your $SCRATCH/eb/mods/all directory to your MODULEPATH so Lmod can find your built software. Nothing else should be required.