Hprc banner tamu.png

Difference between revisions of "SW:cctools"

From TAMU HPRC
Jump to: navigation, search
(initial version)
 
(add references and PARROT_CVMFS_ALIEN_CACHE env var)
Line 8: Line 8:
 
[https://ccl.cse.nd.edu/software/parrot/ Parrot] is a tool for attaching existing programs to remote I/O systems through the filesystem interface.
 
[https://ccl.cse.nd.edu/software/parrot/ Parrot] is a tool for attaching existing programs to remote I/O systems through the filesystem interface.
 
Parrot "speaks" a variety of remote I/O services include HTTP, FTP, GridFTP, iRODS, CVMFS, and Chirp on behalf of ordinary programs.
 
Parrot "speaks" a variety of remote I/O services include HTTP, FTP, GridFTP, iRODS, CVMFS, and Chirp on behalf of ordinary programs.
 +
 +
Parrot cache dir is set to /scratch/user/YourNetID/parrot_cache.  User can change it's location by updating environment variable ''PARROT_CVMFS_ALIEN_CACHE'' prior running "parrot_run" command
 +
export PARROT_CVMFS_ALIEN_CACHE=/scratch/user/YourNetID/my_favorite_cache_dir
  
 
==== Access Parrot ====
 
==== Access Parrot ====
Line 53: Line 56:
 
  (Parrot) [user@terra1 ~]$ '''exit'''
 
  (Parrot) [user@terra1 ~]$ '''exit'''
 
  logout
 
  logout
 +
 +
=== References ===
 +
* CCL software page: https://ccl.cse.nd.edu/software/
 +
* Parrot User Manual: https://cctools.readthedocs.io/en/latest/parrot/

Revision as of 14:27, 21 February 2020

cctools

The Cooperative Computing Tools (cctools) is a software package for enabling large scale distributed computing on clusters, clouds, and grids. It is used primarily for attacking large scale problems in science and engineering. "parrot_run" can be used to access CVMFS.

Parrot

Parrot is a tool for attaching existing programs to remote I/O systems through the filesystem interface. Parrot "speaks" a variety of remote I/O services include HTTP, FTP, GridFTP, iRODS, CVMFS, and Chirp on behalf of ordinary programs.

Parrot cache dir is set to /scratch/user/YourNetID/parrot_cache. User can change it's location by updating environment variable PARROT_CVMFS_ALIEN_CACHE prior running "parrot_run" command

export PARROT_CVMFS_ALIEN_CACHE=/scratch/user/YourNetID/my_favorite_cache_dir

Access Parrot

Parrot is available on Terra cluster and it's working on both login nodes and compute nodes. To access parrot, user can load cctools module

module load cctools/7.0.22-Python-2.7

Alternatively, user can load cctools with Python 3.6 toolchain via

module load cctools/7.0.22-Python-3.7

Parrot examples

In the following examples, the commands are listed in bold face.

User can list CVMFS file system using "parrot_run"

[user@terra1 ~]$ parrot_run ls -lad /cvmfs/cms.cern.ch/slc6_amd64_gcc530
LibCvmfs version 2.4, revision 25
drwxr-xr-x 1 root root 44 Dec 19 05:10 /cvmfs/cms.cern.ch/slc6_amd64_gcc530

User can get a shell with "parrot_run". Please ignore the error of "bash: /usr/bin/mount: Permission denied"

[user@terra1 ~]$ parrot_run bash
2020/02/19 15:03:46.07 parrot_run[17105] <child:17149> notice: warning: system call 316 (renameat2) not supported for program /usr/bin/mv
2020/02/19 15:03:46.38 parrot_run[17105] <child:17178> notice: cannot execute the program /usr/bin/mount because it is setuid.
bash: /usr/bin/mount: Permission denied
[user@terra1 ~]$ ls -l /cvmfs/atlas.cern.ch
LibCvmfs version 2.4, revision 25
total 1
drwxr-xr-x 1 root root 3 Dec  5 08:41 repo

User will get a warning message when running "parrot_run" inside "parrot_run bash"

[user@terra1 ~]$ parrot_run bash
sorry, parrot_run cannot be run inside of itself.
version already running is 7.0.22 FINAL.


User can check if "parrot_run" is running via "parrot_run --is-running" command. If "parrot_run" is not running, no output will be given, otherwise version number will be printed.

[user@terra1 ~]$ parrot_run --is-running
7.0.22 FINAL

User change prompt for parrot_run to be less confusing

[user@terra1 ~]$ PS1="(Parrot) [\u@terra1 \W]\$ " parrot_run bash --login --noprofile
(Parrot) [user@terra1 ~]$ ls -l /cvmfs/atlas.cern.ch
LibCvmfs version 2.4, revision 25
total 1
drwxr-xr-x 1 root root 3 Dec  5 08:41 repo
(Parrot) [user@terra1 ~]$ exit
logout

References