===========
Use scratch
===========

A scratch filesystem is a **temporary** space where you can **copy** input data and write intermediate, temporary and output results from a job. It is not mandatory to work with a scratch but it offer some advantages:


- larger space compared to ``$HOME`` or group-shared datasets (``/Xnfs/$GROUP``),
- provide greater IOPS [#iops]_

However, scratches are **shared spaces**. Their performances depend on good use. They should **NOT** contain:

- documentations (too much small files, and... why?) -> ``$HOME``, ``/Xnfs/$GROUP``
- symbolic links (very tiny files, useless IO)
- source codes or softwares (small files, useless IO) ->  ``$HOME``, ``/Xnfs/$GROUP``
- conda, modules or libraries, virtualenv...  ->  ``$HOME``, ``/Xnfs/$GROUP``

.. HINT:: **Only input data, temporary data and output data.**

.. WARNING:: 
    Scratches are not meant for *long duration storage*. You should cleanup as fast as your job finishes. 

    **DO NOT STORE** source codes (including conda, virtualenv...), small files or symbolic links in ``$SCRATCH``. **It degrade performances very fast, FOR EVERYONE**.

    When scratches are full, PSMN's Staff will erase files and directories blindly.

    **You have been warned.**


Two types of scratches are available, global to a partition (or cluster) or local to a node. See :doc:`../clusters_usage/login_nodes` and :doc:`../clusters_usage/partitions_overview` for access, repartition and paths.

+---------------+-----------+-----------+--------+--------------+---------------------------------------------+
| Name          | Type      | snapshots | quotas |  Performance | Purpose                                     |
+===============+===========+===========+========+==============+=============================================+
| ``$SCRATCH``  | glusterfs |        no |     no |         high |         large temporary files, checkpoints, |
|               |           |           |        |              |                  raw temporary input/output |
+---------------+-----------+-----------+--------+--------------+---------------------------------------------+
| local scratch | ext4, zfs |        no |     no | medium, high | job specific output requiring IOPS [#iops]_ |
| (ssd, disk)   |           |           |        |              | (120 days lifetime residency)               |
+---------------+-----------+-----------+--------+--------------+---------------------------------------------+

.. WARNING:: **DO NOT STORE** source codes (including conda, virtualenv...), small files or symbolic links in ``$SCRATCH``. **It degrade performances very fast, FOR EVERYONE**.

On this PSMN's networks topology, you can visualize how scratches connect to clusters:

.. figure:: /_static/PSMN_network_synoptic_a3_2022d.png
    :alt: PSMN network synoptic 2022
    :width: 660px
    :align: center

    PSMN network synoptic (as of 2022)

Examples
========

Here's two ways of using scratch:

* manual copy

    From a login node, you can create, copy, delete into ``/scratch/$CLUSTER/$USER/whatever/``, before and after a (set of) job(s).


* Automated copy

    Inside your batch script, you can copy input data to scratch, indicate to your software which ``$TMPDIR`` or ``$SCRATCHDIR`` to use. **Also cleanup at the end of a successfull job**.

    This is mandatory for local scratch, on specific nodes. See :doc:`../clusters_usage/partitions_overview` for paths.

See our `repository of examples scripts <https://github.com/ltaulell/submission_scripts>`_.

If you do not feel comfortable with scripts and scratch, `please ask us around a coffee <http://www.ens-lyon.fr/PSMN/doku.php?id=contact:forms:cafe>`_.

.. note::
    These machines were set up thanks to the preparatory work, recipes and integrations carried out on the :term:`CBP` experimental platform.

.. [#iops] See :term:`IOPS`