Documentation

Accessing the cluster

Please keep in mind you need a Google account for these steps.
The procedure to access the cluster is the following:

1. Request an account by filling out our registration form
2. Once the account is approved, log into GitLab using the C3S Unified Login option
3. Upload your public SSH key to your account in the corresponding settings page
4. Login through SSH at username@mercurio.hpc4ai.unito.it
5. With the same account you can log into the web calendar to book the system resources (more details below)

Cluster architecture

Storage

• BeeGFS: 75 TB of high-performance storage is provided and mounted as the users' home area, ensuring smooth access to user data and project resources.


• Lustre: 20 TB filesystem available under /lustre/scratch as a dedicated scratch area, accessible only from compute nodes equipped with OmniPath interconnects. Suitable for temporary data, large computations, and I/O-intensive workloads

BookedSlurm

We have developed a custom Slurm plugin, BookedSlurm, which provides integration between Slurm and a web-based scheduling system called Booked. This integration allows users to reserve computational resources in advance and have usage accounted through credits, the paid currency used by Booked.

For each type of computational resources, two corresponding Slurm partitions exist:

• standard partition — named after the resource
  • open to all users
  • maximum job runtime: 6 hours
  • no reservation required


• booked partition — with the suffix -booked
  • accessible only through a reservation
  • maximum job runtime depending on reservation duration

By logging to the Booked calendar, you will find a complete list of all available nodes. By clicking the “Reserve” button it's possibile to book the resources needed for a certain amount of 2-hours time slots, allowing users to run jobs lasting more than 6 hours. Every resource has a specific cost related to its CPU performance, available memory and the number and type of GPUs. The reserved resources will appear in Slurm exclusively for the reservation owner for the entire duration of the booking.

To submit a job that uses your Booked reservation, you must specify two flags:

• --partition={partition_name}-booked — if you don't specify the “-booked” suffix the jobs will have a maximum duration of 6 hours. You can list the available partition with the Slurm command sinfo


• --reservation={reservation_name} — you have to specify the name of the reservation you created on Booked. You can check the reservations available with the Slurm command scontrol show res

Any job still running after the reservation ends will be automatically cancelled by Slurm. Ensure that your reservation duration fully covers your job's expected runtime.

Services

• Spack

  You can install Spack by running spack_setup and following the prompts provided by the installation script.

  To update Spack and its package repositories:

  • check for the latest Spack release on the official GitHub page (e.g., v1.1.0)
  • go to your Spack installation directory (default is ~/spack)
  • update the repository and switch to the desired release tag: git pull -t && git checkout tags/v1.1.0

  The following example demonstrates how to create a Spack environment (thanks to Flavio Sartori):

  spack env create <my_env>
  spack env activate <my_env>
  spack add python
  spack add openslide
  spack add cuda
  spack concretize
  spack install
  spack load python
  spack load openslide
  spack load cuda
  

• Singularity

  You can run software using Singularity, a container platform designed for high-performance computing (HPC) and scientific workloads. It enables users to bundle applications, libraries, and entire environments into portable containers that behave consistently across different Linux systems.

  Since building Singularity images requires root privileges, it is only possible to run pre-built images on this system. If you need to build your own images for another architecture (e.g., aarch64) on your system, please follow the procedure below (based on Ubuntu):

  # Install needed packages:
  sudo apt install qemu-user qemu-user-binfmt
  
  # Set binfmt interpreters:
  sudo singularity run docker://multiarch/qemu-user-static --reset -p yes
  
  # Get base image for another arch:
  singularity pull --arch aarch64 aarch64-ubuntu.sif library://ubuntu:22.04
  
  # Test if it works
  singularity exec aarch64-ubuntu.sif uname -m
  
  # Build your image locally:
  sudo singularity build aarch64-image.sif your-def.def
  

  An example of a .def file using an aarch64 image:

  Bootstrap: localimage
  From: aarch64-ubuntu.sif
  
  %post
          apt -y update
          apt -y install curl
  
  %runscript
          echo "Hello from $(uname -m)
  

• Slurm REST API

  Slurm REST API are available at: https://api.mercurio.hpc4ai.unito.it

  You can find the official documentation here: Slurm REST API

  Here is a basic usage example: Quickstart

Help desk

For any request or issue, such as the installation of additional software, you can send an email to support@hpc4ai.unito.it