Jupyter notebook

Direct notebook

A Jupyter instance can be launched on any Linux machine that runs Python and has the Jupyter package installed. This includes CHPC's built Python packages (python/2.7.11,  python/3.5.2  and python/3.6.3 modules), Anaconda (anaconda modules) or Intel Python (intelpy modules).

This approach is recommended for those with dedicated interactive nodes or desktops who want the choice of the Python (or other language interpreters in the notebook), or for lightweight programming on general interactive nodes.

To get the Jupyter notebook running:

1. Use FastX to open a remote desktop session on an interactive node (e.g. frisco1.chpc.utah.edu).
2. Open a new terminal window, and there load a Python module and launch the notebook. (Note: the "--browser=" flag allows you to select the web browser to use. Google Chrome works reliably with all versions of python in both the general environment (clusters notchpeak, kingspeak, lonepeak, and ash) as well as the protected environment (redwood cluster). Other browsers such as Firefox and Konqueror frequently generate errors, or fail to connect to jupyter):

   $ module load python3
   $ jupyter notebook --browser=/usr/bin/google-chrome
   [I 14:05:23.164 NotebookApp] Serving notebooks from local directory: /home/....
   [I 14:05:23.164 NotebookApp] 0 active kernels 
   [I 14:05:23.164 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
   [I 14:05:23.164 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

3. A web browser window opens in the remote desktop with the initial Jupyter session.
4. To finish the session, close the web browser and Control-C the Jupyter application in the terminal.

In order to use other interpreters than Python in the notebook, the appropriate module needs to be loaded. E.g. to use R, also module load R/3.4.2., or for Matlab, module load matlab/R2018a.

Notebook with SSH tunelling

Another possibility to run a Jupyter notebook as a job is to launch an interactive job, set up an SSH tunnel to the compute node that the job runs on from a client machine, and then run the notebook in a web browser of the client machine. This works on all CHPC clusters but requires more work to set up than the OnDemand route.

1. Open a FastX remote desktop to a cluster interactive node (e.g. lonepeak1) and open two terminal windows.
2. In a first terminal window, start an interactive job

   [lonepeak1]$ salloc --time=1:00:00 --ntasks 1 --nodes=1 --account=chpc --partition=lonepeak 

3. Once job starts and terminal prompt is back, record the host name of the node where the job runs

   [lp064]$ hostname -f
   lp064.lonepeak.peaks

4. Start the Jupyter notebook on the compute node

   [lp064]$ export XDG_RUNTIME_DIR=""
   [lp064]$ ml python3
   [lp064]$ jupyter notebook --no-browser --port=8889

5. Now switch to the other terminal window (on the interactive node), and start the Google Chrome browser in the background. As of July 2019, we have seen problems with using Firefox
   

   [lonepeak1]$ google-chrome &

6. Start the SSH tunnel (on the interactive node)

   [lonepeak1]$ ssh -N -L localhost:8888:localhost:8889 lp064.lonepeak.peaks

7. In the web browser, type http://localhost:8888/tree to connect to the remote notebook
8. To cancel the session, press ctrl-c in both terminal windows. One will cancel the notebook and the other will cancel the ssh tunnel.

Jupyterhub notebook

Jupyterhub is a multi-user version of Jupyter notebook, a web application that provides an environment that enables users to edit and run code from a browser and to attach the media representation of results (HTML,LaTeX, PNG, PDF, etc..) with the code that generates them. CHPC has retired the two  instances of a Jupyterhub server  (notebook.chpc.utah.edu and juno.chpc.utah.edu) that we used to run, there are alternatives.

For people without a CHPC account, we suggest looking at Jupyter notebook hosting alternatives such as  https://mybinder.org, which we are using for CHPC’s Introduction to Python training series, and https://colab.research.google.com, which has been used by faculty in the Department of Engineering.

For CHPC users who want to run a Jupyter notebook with access to their CHPC file systems, please either use the direct notebook option mentioned above or use the CHPC ondemand server, ondemand.chpc.utah.edu. For more information see our ondemand webpage.

Jupyterhub via ondemand can be used for classroom support.  If interested, please contact CHPC via helpdesk@chpc.utah.edu and we can make arrangement for class accounts.

Using Jupyter Notebook

After starting the notebook you will see your home directory. To open a new notebook, use the New button and choose the appropriate language.

In the new notebook, write your code, e.g.

 

For a short list of Jupyter Notebook commands and features, see our Reference Guide.

 

Using Jupyter with Anaconda Environments

It is not unusual to install python code and Jupyter notebooks as part of an Anaconda environment, or to develop new python code and notebooks within an Anaconda environment, but sometimes it is unclear how to make this code work with your Jupyter installation. The trick to make Jupyter work with an Anaconda environment is to install the ipykernel package within the environment, and then to register that kernel with the Jupyter system. Follow these steps to do this:

1. Use conda to activate your environment: conda activate your_env_name
2. Use conda again to install ipykernel: conda install ipykernel
3. Run python to register this Anaconda environment as a Jupyter kernel: python -m ipykernel install --user --name your_env_name --display-name "Python (your_env_name)"

If you then re-start your Jupyter session, the kernel or kernels you have registered will appear in the "New" drop-down menu.

Converting Jupyter Notebooks to Other Formats

The Jupyter Lab and Notebook environments can save and export notebook files to other formats (for example PDF) but this process is not completely reliable. A more reliable technique is to use the quarto and texlive modules to perform the conversion. For example, to convert a notebook file named "my_homework.ipynb" to PDF, open a terminal window, change directory to the location of the notebook file, and issue the following commands:

module load quarto texlive
quarto render my_homework.ipynb --to pdf

Quarto will convert the document first to LaTex and then to a PDF file named "my_homework.pdf".