Hprc banner tamu.png


Revision as of 08:39, 15 August 2018 by J-perdue (talk | contribs) (Create a sanbox)
Jump to: navigation, search


Prebuilt images =

HPRC provides (will provide) images using the steps below on ada/terra in /scratch/datasets/Singularity/images if you want to use a prebuilt image.

Fedora 28

Note that the final image requires a kernel newer than the one provided by CentOS6. This means it can not be used on ada (only on terra). Fedora version 23-28 were attempted but they eventually failed on ada. So if you plan to use a Fedora-based Singularity image you will need to use terra.


The objective in this case is a Fedora28-HPRCLAB.img file that has space for users to manuever around and install stuff:

  1. that we can provide to users, a Singularity .img file (self-contained), for them to update as needed (as root on their own workstations) that also runs on our clusters.
  2. is already populated with HPRCLAB RPMs
  3. has plenty of disk space for the user to add files (as non-root) while on the cluster
  4. Example below was created on HPRCLAB workstations.

Building an image on your Fedora 28 workstation

Create a "sandbox"
  • start a root shell, but maintain location (you will need it for most of the next steps so might as well do it now to avoid all the password retypes for sudo)
sudo su
  • create a Singularity sandbox from one of Fedora's Docker images
singularity build --sandbox Fedora28-HPRCLAB.sandbox docker://fedora:28
  • create a list of packages to install into the image and place in /root. In this case we use the HPRCLAB/Fedora28 workstations current list of software. So on the workstation/host (not in the image):
rpm -qa --qf "%{NAME}\n" | sort > /root/rpmlist

which results in something that looks like this:

[root@hprclab4 ~]# head -5 /root/rpmlist 
  • while still in the host sudo/root shell update the image using the --writable shell in the sandbox (requires sudo for next steps)
singularity shell --writable Fedora28-HPRCLAB.sandbox/
  • Install the RPMFUSION repos (needed for below)
dnf -y install\
  • populate the sandbox with all the RPMs (we do them in groups at first in case there a problems and then do all to see what those problems were. (get a snack, take a walk... it can take awhile)
for x in {a..z} ; do
  dnf -y install `grep ^$x /root/rpmlist`
dnf -y install `cat /root/rpmlist`

or maybe

for x in `cat /root/rpmlist` ; do
  dnf -y install $x

There will still be packages that have "No match" due to not including the repos for them.

  • do NOT add users with useradd... if you do, be sure not to add your NetID or things will break.
  • create bind directories for interfacing with HPRC clusters (Note: have to do this now while still in sudo shell)
mkdir /scratch /work
  • once you are happy with the sandbox log out from it (logout/exit/Ctrl-D).
Create an image (.img)
  • to make an .img file that provides space for the user to work in (e.g. download/install software). These steps do not require sudo since they don't really create anything in the image.
  • first, determine the installed size of the sandbox
* Given that, add some space for the user to work in.  In this case, we will use 40GB (approx. sandbox size + 15G).
singularity image.create Fedora28-HPRCLAB-40GB.img # create .img file
singularity image.expand -s 40960 Fedora28-HPRCLAB-40GB.img # expand to 40GB, get another snack while you wait

Note: works because there is no real filesystem yet [some ext3/4 filesystems seem to have a problem with expand]

  • Now import the sandbox into the 40GB image (more snacks/time)
sudo tar -cvf - -C Fedora28-HPRCLAB.sandbox/ . | sudo singularity image.import Fedora28-HPRCLAB-40GB.img

(see the last/tar example in 'singularity image.import --help')

Test on host build system
  • open a non-sudo shell and see if you can do "things"
singularity shell Fedora28-HPRCLAB-40GB.img

Using your image on HPRC clusters

CentOS 6

Create a sanbox

  • create/start sandbox
sudo singularity build --sandbox CentOS6.sandbox docker://centos:6
sudo singularity shell --writable CentOS6.sandbox/
* create bind points for HPRC cluster filesystems
sudo mkdir /scratch /work
  • install repos for other RPMS
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
  • populate sandbox with RPM installs
for x in `cat /root/HPRCLAB-rpmlist` ; do yum -y install $x ; done
mkdir /scratch /work
  • exit sandbox shell