Skip to content

Node scratch space

Aim: Provide information about when to local/node ‘scratch’ directory.

Target audience: Nikhef users.

Introduction

All systems have a local ‘scratch’ directory. While you think this is /tmp/, it may not be where you expect, so always use $TMPDIR to find it. On Stoomboot nodes for example, you have a scratch directory that is specific for your job, and you have a fresh one for each job. Using $TMPDIR as a pointer to it ensures you’re always in the right spot. Want more resilience or use the same script on your laptop and Stoomboot? Use ${TMPDIR:-/tmp}. On desktop systems, there’s also /localstore that you can use.

Usage

Intended use

Local/node ‘scratch’ directory should be used for temporary and intermediate results. Output from MC jobs that will be merged in larger files before being written to dCache (or /data).

What doesn't belong on the node scratch space

Examples of items that should go elsewhere include resulting data files and ntuples that should be used in subsequent analysis (put those in dCache or /data, since localstore is neither backed up nor resilient, and $TMPDIR is cleaned up once the compute job completes), private data (put that in your home directory, your personal web pages (in your home directory or web space), scripts, code, and software (put those in /project if its generally useful, or your home directory otherwise).

What are the limitations?

This storage is only available on one system, and not shared at all. It is blazing fast, large, can be re-written at will, but … is ephemeral and fragile.