SW:Magenta

Magenta

"Magenta is distributed as an open source Python library, powered by TensorFlow. This library includes utilities for manipulating source data (primarily music and images), using this data to train machine learning models, and finally generating new content from these models."

Magenta on HPRC clusters

Magenta module

Ideally we would rebuild everything needed for Magenta to optimize for our clusters but this is a significant and less than straightforward task. So for now we recommend the Python solution below.

Magenta via an Anaconda virtual environment

A lot of what is described below is probably easily adapted to Anaconda, but we are going to use a plain Python for now

Magenta via a Python virtual environment

In order to get the most out of Magenta you'll need the following modules:

1. cuDNN/7.0.5-CUDA-9.0.176 - the version of Magenta installed by pip strictly requires CUDA 9.0 and cuDNN 7.0
2. an "oldish" Python. Python 3.6 is too new. At the moment we recommend a Python 3.5.2 from the 2017A toolchain.

Verify the needed sound library development files are installed (currently only on ada2 and terra2):

rpm -q alsa-lib-devel

Installing

Using the steps here, and replacing 'venv' with '$SCRATCH/mymagenta' we can create a virtual environment to install magenta-gpu (we want the GPU version given our TensorFlow benchmarks. We'll start with the 2017A, GCC-based, 'foss' toolchain to keep things simple.

ml purge # clean module environment
ml Python/3.5.2-foss-2017A # load an older Python
ml cuDNN/7.0.5-CUDA-9.0.176 # load the needed CUDA/cuDNN
ml # list modules

Those steps will be required everytime you wish to access Magenta. If you load other modules, make sure they all comply with foss-2017A toolchain.

In order to install Magenta, we'll need to use the steps above to create a virtual environment and install Magenta.

rm -rf $SCRATCH/mymagenta # remove previous attempts (???)
virtualenv $SCRATCH/mymagenta # create virtual environment
source $SCRATCH/mymagenta/bin/activate # activate venv
pip install magenta-gpu # install magenta/tensorflow w/GPU

Testing

A quick test is:

1. run the 'ml' commands above to load the needed modules
2. run: python -c 'import magenta; magenta.__version__'