Skip to content

Directory /data

Aim: Provide information about when to use NFS-mount /data.

Target audience: Nikhef users.

Introduction

This is a regular, shared, file system for storing moderately-sized (1-100 Gigabyte) data files that could in principle be re-produced (or downloaded again from the Grid or your data lake). It is ‘NFS’ mounted from a dedicated server in the data center, but the amount of transactions per second (read or write) does not quite scale to the full size of Stoomboot. Use this for private ntuples and results that you use for small analyses, where you need to re-write files, or modify existing files. The amount of storage in /data is limited to several terabyte per user.

Usage

Intended use

/data is for modestly-sized analysis results, software and containers that can be re-created or downloaded again without much effort, Singularity (Apptainer) and docker images that are used as containers, log files that you need to collate and study later. Exceptionally, /data/tunnel is designated to share data between the Nikhef local and public grid environments.

What doesn't belong in /data

Examples of data types that are better put elsewhere: your software and scripts (these should be in /project or your home directory), intermediate log files (these should be in $TMPDIR or the node scratch space).

What are the limitations?

This filesystem is not backed up, so if the system fails catastrophically: too bad. You will have to re-create or re-download the data. The NFS server can handle quite some transactions, but abuse will impact both desktop and Stoomboot users in the same way (so do not "hammer" it). It is a fully ‘POSIX compliant’ file system, with permissions, so (unlike dCache storage) to can modify existing files. Out of space? Ask your group leader to discuss with the CT-PDP team.

Did you know...

Copying data between the /data and dCache storage classes is generally useless? While their intended use is different, once the data is in either place, moving it around only wastes resources. The only exception is if you want to share your results outside of Nikhef (since Stoomboot dCache can be accessed with xrootd or gridftp protocols, and /data cannot) and then only when data is actually being re-written during its creation (otherwise, just write to dCache immediately).