Local Scratch

Using Node Local Storage

When Slurm starts a job, it creates a temporary directory on each node assigned to the job. It then sets the full path name of that directory in an environment variable called TMPDIR.

Because this directory resides on local disk, input and output (I/O) to it is almost always faster than I/O to a network storage (/global/project, /global/scratch, or /global/home). Specifically, local disk is better for frequent small I/O transactions than network storage. Any job doing a lot of input and output (which is most jobs!) may expect to run more quickly if it uses $TMPDIR instead of network storage.

The temporary character of $TMPDIR makes it more trouble to use than network storage. Input must be copied from network storage to $TMPDIR before it can be read, and output must be copied from $TMPDIR back to network storage before the job ends to preserve it for later