|
|
| (39 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| − | == j-perdue random links for 8/10/2018 ==
| + | See [[SW:Singularity:Examples]] |
| − | === Placeholder ===
| |
| − | To be added to SW:Singlularity:Examples or something.... just a placeholder for now to dump all my URLs at home for access at work:
| |
| − | | |
| − | https://github.com/singularityware/singularity/issues/1076
| |
| − | | |
| − | https://github.com/singularityware/singularity/issues/1165
| |
| − | | |
| − | http://singularity.lbl.gov/quickstart#build-images-from-scratch
| |
| − | | |
| − | https://hub.docker.com/_/fedora/
| |
| − | | |
| − | https://github.com/NIH-HPC/singularity-examples/blob/master/torch/Singularity.torch
| |
| − | | |
| − | https://www.sylabs.io/docs/
| |
| − | | |
| − | === history ===
| |
| − | | |
| − | * random notes from .bash_history on hprclab2
| |
| − | | |
| − | <pre>
| |
| − | [j-perdue@hprclab2 ~]$ history | grep docker
| |
| − | 460 git clone https://github.com/fedora-cloud/docker-brew-fedora.git
| |
| − | 466 less prep-docker-brew-branch.sh
| |
| − | 468 ./prep-docker-brew-branch.sh 28
| |
| − | 470 sudo docker pull fedora:28
| |
| − | 472 sudo docker run --rm -ti fedora:28 bash
| |
| − | 484 sudo docker run -ti fedora:28 bash
| |
| − | 494 sudo docker run -ti fedora:28 bash
| |
| − | 495 man docker
| |
| − | 496 man docker-run
| |
| − | 497 sudo docker build --writable Fedora28.img fedora:28
| |
| − | 515 sudo singularity build --writable FedoraLatest.img docker://fedora:latest
| |
| − | 516 sudo singularity build --writable Fedora28.img docker://fedora:28
| |
| − | 523 # sudo singularity build --writable --size Fedora28.img docker://fedora:28
| |
| − | 541 history | grep docker
| |
| − | [j-perdue@hprclab2 ~]$ history | grep singularit
| |
| − | 510 sudo singularity build --writable Fedora28.img fedora:28
| |
| − | 511 sudo singularity build --writable Fedora28.img fedora:27
| |
| − | 512 sudo singularity build --writable Fedora28.img
| |
| − | XXX sudo singularity shell --writable Fedora28-HPRCLAB.img
| |
| − | </pre>
| |
| − | | |
| − | === nearing in on objective ===
| |
| − | | |
| − | The objective in this case is a Fedora28-HPRCLAB.img file that has space for users to manuever around and install stuff:
| |
| − | # that we can provide to users, a Singularity .img file (self-contained), for them to update as needed (as root on their own workstations) that also runs on our clusters.
| |
| − | # is already populated with HPRCLAB RPMs
| |
| − | # has plenty of disk space for the user to add files (as non-root) while on the cluster
| |
| − | # Example below was created on HPRCLAB workstations.
| |
| − | | |
| − | ==== Latest ====
| |
| − | | |
| − | * create a Singularity sandbox from one of Fedora's Docker images
| |
| − | <pre>
| |
| − | sudo singularity build --sandbox Fedora28-HPRCLAB.sandbox docker://fedora:28
| |
| − | </pre>
| |
| − | * create a list of packages to install into the image and place in /root. In this case we use the HPRCLAB/Fedora28 workstations current list of software. So on the workstation (not in the image):
| |
| − | <pre>
| |
| − | sudo su - # need root for redirect
| |
| − | rpm -qa --qf "%{NAME}\n" | sort > /root/rpmlist
| |
| − | </pre>
| |
| − | which results in something that looks like this:
| |
| − | <pre>
| |
| − | [root@hprclab4 ~]# head /root/rpmlist
| |
| − | a2ps
| |
| − | a52dec
| |
| − | aajohan-comfortaa-fonts
| |
| − | aalib-libs
| |
| − | aalto-xml
| |
| − | abattis-cantarell-fonts
| |
| − | abrt-addon-coredump-helper
| |
| − | abrt-gui-libs
| |
| − | abrt-libs
| |
| − | accountsservice
| |
| − | </pre>
| |
| − | * open a sudo --writable shell in the sandbox (requires sudo for next steps)
| |
| − | <pre>
| |
| − | sudo singularity shell --writable Fedora28-HPRCLAB.sandbox/
| |
| − | </pre>
| |
| − | * populate the sandbox with all the RPMs (we do them in groups at first in case there a problems and then do all to see what those problems were. (get a snack, take a walk... it can take awhile)
| |
| − | <pre>
| |
| − | for x in {a..z} ; do
| |
| − | dnf -y install `grep ^$x /root/rpmlist`
| |
| − | done
| |
| − | dnf -y install `cat /root/rpmlist`
| |
| − | </pre>
| |
| − | * add users
| |
| − | <pre>
| |
| − | useradd mynetid
| |
| − | </pre>
| |
| − | * once you are happy with that, time to make an .img
| |
| − | * singularity image.create my20Gimage.img # create .img file
| |
| − | * singularity image.expand -s 20480 my20Gimage.img # expand to 20GB (works because there is no real file system yet)
| |
| − | * sudo tar -cvf - -C mysandbox/ . | sudo singularity image.import my20Gimage.img # see the last/tar example in 'singularity image.import --help'
| |
