SW:EasyBuild:Building Your Own Software:Another Example - LLVM
This example will cover building a very recent version of LLVM using the CLang compiler. Users may first want to visit HPRC's EasyBuild page and a previous example of modifying an existing easyconfig. At some point in the process, users might want to also review the official EasyBuild documentation.
- 1 Motivation
- 2 Important note on using EasyBuild to build software on terra
- 3 Converting an existing LLVM EasyBuild "easyconfig" using GCC to a much newer LLVM built using the CLang
- 4 Building the new easyconfig
- 5 Notes
A user wants to build a modified version of the LLVM module that includes:
- a recent source from the LLVM Github repository
- adds support for CLang
LLVM is a rather long build (several hours using a single-processor). Additionally, the currently installed versions do not include CLang support. They are also rather dated (v9.01 released December 20, 2019). Newer releases include v10.0.1, released August 6, 2020. An even newer release candidate includes v11.0.0-rc5 released September 30, 2020 but as of this writing on October 3, 2020, there are already 7681 commits to master since this release. So, for this example we will use a recent (10/3/2020) .zip file of the master repo. Notes for using a bleeding edge (as of this moment) clone of the master repo are included below and are left as an exercise to the reader.
Important note on using EasyBuild to build software on terra
The instructions below are for building on any ada (and previously curie) login node. At present, on terra, the needed directories are only mounted on terra2/login2, so if you want to build for terra you will need to build there. Some things to consider:
- If ada2 is currently, or was recently, down (e.g. for system maintenance), the needed directories may not be available on terra2. Additionally, if terra2 was recently rebooted they may not be available. If ada2 (and terra2) is up and available for users and these directories are not yet available on terra2, please email firstname.lastname@example.org to request they be mounted
- If the build requires a GPU at compile (build) time instead of just run time, you will need to use terra3/login3 (the only terra login node with a GPU). If you find your programs has this compile-time requirement, please email email@example.com and someone should be able to mount the needed directories there. However, for most builds, users should use terra2/login2.
Some additional notes:
- Once you have built your software on terra using EasyBuild and only want to use the software you have built, the status of ada2 or terra2 is of no concern. Any software you build should a be available in your $SCRATCH directory using the 'myEB' module.
- At some point we hope to set up an independent | CVMFS environment for hosting, and building, all our EasyBuild software such as | the one in use up at Compute Canada. This will eliminate our dependency on any particular cluster to build/deploy software. At present, however, we do not presently have the resources to do so (they have a few more people than TAMU HPRC does and it will take a bit of planning and work to do properly here).
Converting an existing LLVM EasyBuild "easyconfig" using GCC to a much newer LLVM built using the CLang
At some point, you may want to write your own easyconfig from scratch for software not yet built on HPRC systems. That is left an exercise to the reader after they have become familiar with using EasyBuild. For now, we will stick to modifying an existing easyconfig that includes some "simple" user modifications.
Selecting an existing LLVM easyconfig
Before we get started, we'll want to find an easyconfig to convert. In this case, the user wants the most recent version of LLVM. The user also desires to use the currently HPRC-recommended 2019b toolchain which is built on GCCcore/8.3.0. See SW:Toolchains for details on toolchains currently used on HPRC clusters.
Load the EasyBuild-cluster-SCRATCH environment
First, load the "SCRATCH" configuration for using EasyBuild on the cluster you are building on. For example:
# purge all modules to start clean # EasyBuild prefers no other modules to be loaded when in use to avoid interference ml purge # load the latest EasyBuild environment to build for your $SCRATCH directory on ada ml EasyBuild-ada-SCRATCH
When done, you should have something like this:
ml # list modules
- the "EasyBuild" module sets default environment variables using the actual EasyBuild software
- the "EasyBuild-ada" module that sets a number of environment variables for use on ada
- the "EasyBuild-ada-SCRATCH" module that sets/overrides environment variables for installing into the user's $SCRATCH directory (/scratch/user/<NetID>)
Search for candidate easyconfigs
To search the files currently available via $EASYBUILD_ROBOTPATH, run:
eb --search LLVM
This will provide a list of easyconfigs either provided by EasyBuild or built on the current cluster
- the files in /sw/eb/sw/EasyBuild/<version>/easybuild/easyconfigs are the ones provided by EasyBuild in their unaltered form
- the files in /sw/eb/ebfiles_repo_cleaned/<cluster> are the files actually used to build the software on the cluster
Usually, if a file occurs in both locations, then have the some contents. However, sometimes we've made changes to the easyconfig to adapt it to the current cluster. For example, for the Python build on ada, we uncomment the line for OpenSSL because the system openssl-devel on ada is too old for the latest Pythons.
We recommend the user start with the official EasyBuild easyconfig and then consult the local easyconfig if they run into problems.
In this case, we will use the official EasyBuild easyconfig of the latest version of the available LLVM easyconfigs using the desired toolchain, which is /sw/eb/software/EasyBuild/4.3.0/easybuild/easyconfigs/l/LLVM/LLVM-9.0.1-GCCcore-9.3.0.eb
Note: on ada, /sw/eb/sw is the same as /sw/eb/software. We (will) use /sw/eb/sw on terra and all new clusters (using "sw" is easier/better than "software", but ada was originally deployed using "software" so we are stuck with it for now)
Copy to a work directory so you can make modifications
mkdir $SCRATCH/tmp # if you don't already have such a directory cp /sw/eb/sw/EasyBuild/4.3.0/easybuild/easyconfigs/l/LLVM/LLVM-9.0.1-GCCcore-9.3.0.eb $SCRATCH/tmp # note we use the shorthand "sw" here cd $SCRATCH/tmp # change to work directory so we can make changes ls -al LLVM* # verify the file was copied
Making the desired changes to the easyconfig
First, rename the file to reflect the version we plan to use.
mv LLVM-9.0.1-GCCcore-8.3.0.eb LLVM-master-GCCcore-8.3.0.eb
Now edit the file using your favorite editor. "vi" is guaranteed to be on every Unix system and is highly recommended but involves a bit of a learning curve if you aren't familiar with it. "gedit" provides a nice GUI editor that should be usable by most users but does require you enabled X11 when you connected. "nano" is a very friendly text-based editor popular with many users. "emacs" is an overly complicated editor with lots of bells and whistles not needed by even the most seasoned user but is available via the module system. (Note: religious wars over "vi vs. emacs" have waged for decades and will likely continue as long as Unix exist. Yours truly (the author of this page) is a long time (30+ years) "vi" user so is obviously biased towards using "vi".)
Begin editing the file with:
vi LLVM-master-GCCcore-8.3.0.eb # replace "vi" as desired