Building Apptainer/Singularity containers locally

Preparing Your Computer

Singularity runs natively on Linux. If you have root access on your Linux machine, you can install Singularity natively and proceed to step 2.

Similarly, if you are working on a MacOS, the Singularity website provides information on how to install a Linux Virtual Machine using Vagrant. Following installation of Singularity with Vagrant, you can proceed to step 2.

If you are working on a Windows machine, we recommend that you install a Linux Virtual Machine on your Windows machine, inside of which Singularity for Linux can be installed. These installation steps on a Windows machine are detailed in step 1.1 below.

Installing a Linux VM on a Windows machine

In order to make the Linux VM creation simple, we will be using a few command line tools - GIT Bash (alternatively, FastX), Virtual Box and Vagrant.

This whole process is nicely described at the Sylabs page, but we will outline the steps below as well, along with a few extra details. So, follow Sylab's recipe along with our added details below.

Download and Install the Prerequisites.

You need to install the following software to your Windows machine: GIT Bash (which is a part of Git for Windows), Virtual Box, and Vagrant. All installs are GUI based and straightforward.

Git Bash provides a simple to install Linux-like terminal for Windows. If you are used to using Linux commands, it is a great way to operate on Windows. You can also use Git Bash to ssh to other Linux machines, but that is useful only for quick checks. For persistent connections and/or to use graphical applications, use FastX.

Once GitBash is installed, right click on the desktop and you should see Git Bash here ..., click on that and a command shell window opens. This new window is a bash shell (with MINGW Linux running underneath). We will be using this bash shell to do the rest of the VM work.

Set up the Linux Ubuntu Virtual Machine

We can do this as any Windows user in Git Bash, but, if you want to sync a Windows folder to your Linux VM, you need to start Git Bash as an administrator. To do this, go to the search bar at the bottom left Windows pane, type Git Bash, right click it, and choose "Run as Administrator". Then run these first few commands (ignoring the '$' as the the $ means the beginning of a command prompt):

$ pwd
/c/Users/Martin

Notice that we are in your User's directory, on drive C:. It is preferable to organize files and directories in a certain fashion, so let's put all our container files into a different directory, e.g.:

$ mkdir -p /c/work/containers/vm_singularity
$ cd /c/work/containers/vm_singularity

Now we are ready to prepare the VM. First, we need to add, initialize, and download the base image for the VM:

$ vagrant init sylabs/singularity-3.1-centos-7-64 --box-version 20190228.0.0

Search for the most recent Singularity Vagrant files at Sylabs Vagrant repository and change the name/version accordingly. In the example above, we are using the base image name 'sylabs/singularity-3.1-centos-7-64' and we are using version '20190228.0.0'.

The above command prepares a file called Vagrantfile, which defines the Ubuntu VM setup.

If we want to sync folders with our Windows host, we first need to create a directory named vmfiles. This directory will map to a directory within the Ubuntu VM in the next step. To create the folder vmfiles, do this:

$ mkdir vmfiles

Now, edit the Vagrantfile and add the following text right before the last line that says end in the Vagrantfile:

  config.vm.synced_folder "vmfiles", "/vagrant", type: "smb"

The code above instructs Vagrant to map the /vagrantfile directory inside the Ubuntu VM to the vmfiles directory that we just created, located at /c/work/containers/vagrant_ubuntu14/vmfiles on your Windows host.

We also need to make sure that the Virtual Box Guest Additions are installed and up to date (guest additions are extra packages for the VM guest - Ubuntu in our case). This can be done by installing a Vagrant plugin:

$ vagrant plugin install vagrant-vbguest

Now we are ready to start the Ubuntu VM:

$ vagrant up

If you set up the folder sync (three steps above in the example code), you will be prompted for your Windows username and password.

If your Guest Additions are not present or out of sync with the Virtual Box, you'll see a lot of messages pop up. These messages are okay to ignore and the VM will eventually start up. If you get any errors, make sure to update Virtual Box and Vagrant to the latest versions and try to start the VM again.

Now we can ssh to the VM and see Singularity:

$ vagrant ssh
$ which singularity
/usr/local/bin/singularity

And that is all you need to do to get a Singularity VM running on your Windows VM! You can now call Singularity commands either as a user (singularity) or as a root user (sudo singularity).

To shut down the VM when you are done, exit from the ssh session and halt the VM by:

$ exit
$ vagrant halt

Building an Apptainer/Singularity Container

The Singularity website offers a host of tutorials on how to use Singularity and how to build containers. Here, we will focus on building an Ubuntu-based Linux container, because, in our experience, most of the harder-to-install packages are developed on Ubuntu.

There are two basic methods to building a Singularity container: Method A is to run the singularity command, which allows you to provision the container and to bootstrap it (bootstrap meaning that singularity will install the OS and required programs into the container). Method B is to tell Apptainer what OS should be installed and how it should be installed. This can either be done in a definition file or by bootstrapping a definition file. Bootstrapping a definition file is done by importing another container definition file, such as Docker. Different ways to import a container are described in detail in this Singularity documentation.

In our simple demonstration here, we will build a container based on a bootstrap definition file. The container we will build should serve as a good base for Ubuntu-based application deployment.

When deploying a locally built container on CHPC machines, one important step is to create mount points to CHPC file spaces, enabling us to use our home directory and scratch file systems inside the container. To create these mount points, edit the bootstrapped definition file to include the following at the end of the %post section:

mkdir /uufs /scratch

 Once you have included the above code in the bootstrapped definition file, you can now create the container. If we want to build a container named ubuntu.sif from a bootstrapped definition file called ubuntu.def, the command to create the container is:

sudo singularity build ubuntu.sif ubuntu.def

There will be a lot of output as the OS and programs are being installed. Once that is done and no error messages have been produced, the container is ready.

To confirm that the container is working, we can shell into it:

singularity shell -s /bin/bash ubuntu.sif

If you can successfully shell into the container, then the container has been created. A next common use case is installing additional applications into the already-built container. If one needs to install additional packages, we need to build a "sandbox" container - which allows writing on the container - and shell into the container as sudo with write permissions (-w):

sudo singularity build --sandbox development ubuntu.def
sudo singularity shell -w -s /bin/bash development

The above will 1) create a sandbox container called 'development' from the ubuntu.def definition file we bootrapped and editing in previous steps and 2) enable write permissions onto the sandbox container to enable application install within a shell. Once shelled into the container, you can install any applications just as you would in a native Ubuntu OS.

After you have interactively installing packages into the sandbox container, copy/paste the installation commands to the %post section of the definition file (ubuntu.def in this case), so that they can be automatically included within any future container build without having to create a sandbox container again. When installing new applications into a container, it is typically good practice to iterate between running the install commands interactively and putting them into the bootstrap definition file until the automatic build works correctly.

Once you have successfully installed the applications into the sandbox container interactively, you can build the production .sif container file by:

sudo singularity build ubuntu_production.sif development/

Deploying a Singularity Container within CHPC Clusters

Once we have a container built to our liking, we need to move it to the CHPC systems if not already there. One can use their favorite scp program (WinSCP, CyberDuck) - if the folder sync between the host and VM guest were set up, or scp the container directly from the Linux VM (or native Linux machine) you are running.

The below command assumes that you have a directory called 'my_containers' within your home directory. Additionally, you will need to replace 'u0123456' with your uNID.

scp ubuntu_production.sif u0123456@kingspeak.chpc.utah.edu:~/my_containers

To test the container, we ssh to a CHPC machine and run:

ml singularity
singularity shell ~/my_containers/ubuntu_production.sif

If you can successfully shell into the singularity container on CHPC machines - congrats! You have successfully built a container using a bootstrapped definition file.