CHPC Deep Learning Module
CHPC encourages users to install and maintain their own copy of python and associated
libraries ( see
https://www.chpc.utah.edu/documentation/software/python-anaconda.php ). However, many of our users require up-to-date deep learning packages in python
for various machine learning projects. Therefore, CHPC has committed to maintaining
a module (named deeplearning) with recent versions of python, tensorflow, keras, and pytorch. Our intent is to
update this as needed every 6 months to keep it relatively current.
GPU support
The deeplearning module loads with CHPC's CUDA module, which is compatible with all of CHPC's GPUs with the exception of the NVIDIA Tesla K80 GPUs found on the kingspeak cluster. Our GPUs are described here: https://www.chpc.utah.edu/documentation/guides/gpus-accelerators.php .
Versions and documentation
| Module/version | Build Date | Python | Tensorflow | Keras | PyTorch | CUDA |
| deeplearning/2023.3 | 03/13/2023 | 3.10.9 | 2.11.0 | 2.11.0 | 1.13.1+cu117 | 11.8.0 |
| deeplearning/2023.1 | 01/05/2023 | 3.10.8 | 2.10.0 | 2.10.0 | 1.13.1+cu117 | 11.2 |
| deeplearning/2022.1 | 01/31/2022 | 3.9.7 | 2.6.2 | 2.6.0 | 1.10.2+cu102 | 11.2 |
Python libraries
The python libraries available through this module include:
- tensorflow
- keras
- torch
- torchvision
- torchaudio
- ipykernel
- numpy
- scipy
- sklearn
- skimage
- seaborn
- pandas
- PIL
Jupyter support
The deeplearning module is implemented with a singularity container based on the docker jupyter/tensorflow-notebook image ( https://hub.docker.com/r/jupyter/tensorflow-notebook ) to which we have added the torch, torchvision, and torchaudio modules. As such, this module provides support for both jupyter lab and jupyter notebook, making it compatible with our Open OnDemand web portal ( https://www.chpc.utah.edu/documentation/software/ondemand.php ). To use the latest deeplearning module for OnDemand jupyter sessions, select "CHPC Deep Learning 2023.3" from the "Jupyter Python version menu".
Use in scripts or on the command line
To use the deeplearning module in scripts or on the command line, load the module and execute python:
module load deeplearning
python
That version of python is executed from within the singularity container and includes the libraries listed above, the python standard library, and others.
Examples of use
TensorFlow Examples
TensorFlow examples are available at https://github.com/tensorflow/examples and can be installed like this:
$ module load git$ git clone https://github.com/tensorflow/examples.git
Cloning into 'examples'...
...
Updating files: 100% (2762/2762), done.
Keras Examples
There are many Keras examples at https://keras.io/examples/
PyTorch Examples
A variety of PyTorch examples are available at https://github.com/pytorch/examples.git . They can be installed like this:
$ module load git
$ module load deeplearning/2023.3
$ git clone https://github.com/pytorch/examples.git
Cloning into 'examples'...
...
Resolving deltas: 100% (1903/1903), done.Some of the examples require futher installation before they can be run, for example:
$ cd examples/reinforcement_learning
$ pip install --user -r requirements.txt
Note the use of the "--user" flag. This is required to direct the installation into the user's home directory, rather than into the container from which python is run (which is a read-only image).
Container construction
The files used to construct the container for this module are located here: https://github.com/CHPC-UofU/deeplearning-module
Feedback or additions to the module
We welcome feedback from you about this module. If you have any comments or would like additional libraries to be installed in future versions of our deeplearning module please write to us at helpdesk@chpc.utah.edu .