Skip to content

Containers

Aim: Provide information about how to use containers on Nikhef's computing infrastructure to run different operating systems.

Target audience: Users of the Stoomboot cluster.

Introduction

Containers allow you to run another operating system distribution (i.e., linux distributions like CentOS, Ubuntu, etc.) inside a server. Containers are very useful especially if

  1. your collaboration/experiment provides containers for performing analysis runs (usually made available by CernVMFS).
  2. the analysis has specific operating system requirements, i.e., CentOS 7 or another OS distribution.

Nikhef does not allow superuser privileges (aka root priveleges) on Nikhef machines (except our laptops). Apptainer in user mode is the preferred container solution.

Prerequisites

  • A Nikhef account;
  • An ssh client.

Usage

Getting started with Apptainer

The Apptainer binary is available on CVMFS from the Open Science Grid: /cvmfs/oasis.opensciencegrid.org/mis/apptainer/current/bin. It makes sense to add the path to the Apptainer binary to your path to avoid very long commands in your scripts.

export PATH=$PATH:/cvmfs/oasis.opensciencegrid.org/mis/apptainer/current/bin

Apptainer downloads images to a cache directory, which can quickly fill up your home directory (the default cache directory). To avoid this, set APPTAINER_CACHEDIR to a path on /data

export APPTAINER_CACHEDIR=/data/my_group/my_username/.apptainer_cache

Adding directories to the container

To access directories outside the working directory, Apptainer can be told to bind-mount the external directory using the -B argument. The value is a comma-separated list of outside_directory:inside_directory:

> apptainer run -B /project/my_project/my_username:/project,/dcache:/dcache docker://ubuntu:latest
INFO:    Using cached SIF image
INFO:    Converting SIF file to temporary sandbox...
WARNING: underlay of /etc/localtime required more than 50 (66) bind mounts
Apptainer> ls /dcache
alice  antares  atlas  auger  bfys  datagrid  detrd  etseis  gravwav  hisparc  km3net  test  theorie  xenon

Running the Container

Apptainer supports running docker containers:

> apptainer run docker://ubuntu:latest
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob 7b1a6ab2e44d done
Copying config e7132beceb done
Writing manifest to image destination
Storing signatures
2021/12/03 13:36:54  info unpack layer: sha256:7b1a6ab2e44dbac178598dabe7cff59bd67233dba0b27e4fbd1f9d4b3c877a54
INFO:    Creating SIF file...
INFO:    Converting SIF file to temporary sandbox...
WARNING: underlay of /etc/localtime required more than 50 (66) bind mounts
Apptainer> pwd
/user/my_username
Apptainer> cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Repeatedly Running Containers

To submit a custom container to the cluster this first needs to be converted to a .sif file, this to prevent exesive network usage, or hitting rate limits within dockerhub or other registeries.

To speed up the start of Apptainer by avoiding repeated builds of the containers, you can convert an image:

> apptainer pull /data/my_group/my_username/ubuntu_latest.sif docker://ubuntu:latest

And then start it with

> apptainer run /data/my_group/my_username/ubuntu_latest.sif

The --writable argument can be passed to apptainer run, which will allow installation of additional software in the image. Important caveat: the installation of software should not try to change the user (which is not allowed).

Building and installing software in shared directories is possible, but in this case, the path inside and outside of the container should be the same to avoid issues with relocation of built libraries and hard-coded paths.

More detailed instructions on how to build a container can be found at https://apptainer.org/docs/user/main/build_a_container.html.

Adding GPU resources to the container

To get access to the GPUs in the system, pass either --rocm or --nv to apptainer run as needed. This requires the installation of the runtime inside the container. And it build with --fix-permissions to make sure one has access to the relevant interfaces.

Contact

  • Email stbc-admin@nikhef.nl for help using containers.