Computing resources

Our clusters are grouped as partitions, by CPU generations, RAM size and infiniband networks:

  • Big picture:

Partition

CPU family

Network

main Scratch

Best use case

E5

E5

56Gb/s

/scratch/E5N

sequential, small parallel

E5-GPU

E5

56Gb/s

none

sequential, small parallel , GPU computing

Lake

Sky Lake

56Gb/s

/scratch/Lake

medium parallel, sequential

Cascade Lake

Cascade

Cascade Lake

100Gb/s

/scratch/Cascade

large parallel

Epyc

AMD Epyc

100Gb/s

/scratch/Lake

large parallel

See Partitions overview for more details.

Available resources

Use the sinfo 1 command to view the list of partitions (default one is noted with a ‘*’) and their state (also sinfo -l and sinfo -lNe):

$ sinfo
PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
E5*          up 8-00:00:00      4   idle c82gluster[1-4]
Cascade      up 8-00:00:00     77   idle s92node[02-78]

To see more informations (cpus and cpu organization, RAM size [in MiB], state/availability), use one of these:

$ sinfo -e -o '%9P %.8z %.8X %.8Y %.8c %.7m %.5D %N'
PARTITION    S:C:T  SOCKETS    CORES     CPUS  MEMORY NODES NODELIST
E5*          2:8:1        2        8       16  128872     4 c82gpgpu[31-34]
E5*          2:8:1        2        8       16   64328     3 c82gluster[2-4]
E5-GPU       2:4:1        2        4        8  128829     1 r730gpu20
Lake        2:16:1        2       16       32  385582     3 c6420node[172-174]
Cascade     2:48:1        2       48       96  385606    77 s92node[02-78]

$ sinfo -e -o '%9P %.8c %.7m %.5D %.14F %N'
PARTITION     CPUS  MEMORY NODES NODES(A/I/O/T) NODELIST
E5*             16  128872     4        3/1/0/4 c82gpgpu[31-34]
E5*             16   64328     3        3/0/0/3 c82gluster[2-4]
E5-GPU           8  128829     1        0/1/0/1 r730gpu20
Lake            32  385582     3        1/2/0/3 c6420node[172-174]
Cascade         96  385606    77     47/26/4/77 s92node[02-78]

$ sinfo -e -o '%9P %.8c %.7m %.5D %.20C'
PARTITION     CPUS  MEMORY NODES        CPUS(A/I/O/T)
E5*             16  128872     4           13/51/0/64
E5*             16   64328     3           35/13/0/48
E5-GPU           8  128829     1              0/8/0/8
Lake            32  385582     3           16/80/0/96
Cascade         96  385606    77   3808/3200/384/7392
$ sinfo -lN | less
NODELIST     NODES PARTITION       STATE CPUS    S:C:T MEMORY TMP_DISK WEIGHT AVAIL_FE REASON
[...]
c82gluster4      1       E5*        idle 16      2:8:1  64328        0      1   (null) none
s92node02        1   Cascade        idle 96     2:48:1 385606        0      1   (null) none
[...]

Important

  • HyperThreading 2 is activated on all Intel nodes, but not available as computing resources (real cores vs logical cores).

  • RAM size is in MiB, and you cannot reserve more than 94% of it, by node.

Basic defaults

  • default partition: E5

  • default time: 10 minutes

  • default cpu(s): 1 core

  • default memory size: 16GiB / core

Maximums

Here are some maximums of usable resources per job:

  • maximum wall-time : 8 days (‘8-0:0:0’ as ‘day-hours:minutes:secondes’)

  • maximum nodes per job and/or maximum cores per job:

Partition

nodes

cores

gpu

E5

24

384

E5-GPU

20

80

20

Lake

24

768

Epyc

14

1792

Cascade

76

7296

1

You can get the complete list of parameters by referring to the sinfo manual page (man sinfo).

2

See HyperThreading