Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The computational resources of Talapas are divided into groups of nodes called partitions. A partition needs to be specified when submitting a job to tell the workload manager Slurm where to run the job.

Partitions are separate queues for submitted jobs and each partition has different resources and constraints that the user should be aware of when submitting a job.

The tables below lists Slurm partitions on Talapas and . This includes resources and characteristics of the nodespartitions with resources that you may not have access to (you’re not a member of the PIRG) but the resources are available through the preempt partition.

partitions

Code Block
# short, long
NodeName=n[001-047] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=117595 Feature=broadwell,e5-2690
NodeName=n[048] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=180771 Feature=broadwell,e5-2690
NodeName=n[049-096] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=117595 Feature=broadwell,e5-2690

# gpu, longgpu, testgpu
NodeName=n[097,099-113,115-118,120] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=246385 Feature=broadwell,e5-2690,k80 Gres=gpu:tesla_k80:4
NodeName=n[114] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=246385 Feature=broadwell,e5-2690,k80 Gres=gpu:tesla_k80:2
NodeName=n[119] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=246385 Feature=broadwell,e5-2690,k80 Gres=gpu:tesla_k80:3

# fat, longfat
NodeName=n[121-124] Sockets=4 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=2052811 Feature=broadwell,e7-4830
NodeName=n[125-126] Sockets=4 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=4117194 Feature=broadwell,e7-4830
NodeName=n[127-128] Sockets=4 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=1020522 Feature=broadwell,e7-4830

# jhp
NodeName=n[129-136] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=246385 Feature=broadwell,e5-2690

# interactive, steck
NodeName=n[137-139] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=117595 Feature=broadwell,e5-2690

# racs, racsepyc
NodeName=n[140] Sockets=2 CoresPerSocket=14 ThreadsPerCore=1 RealMemory=180771 Feature=broadwell,e5-2690
NodeName=n[224] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6148
NodeName=n[269] Sockets=2 CoresPerSocket=32 ThreadsPerCore=1 RealMemory=504416 Feature=epyc,a100 Gres=gpu:3g.20gb:2,gpu:nvidia_a100_80gb_pcie:1

# dufek, paty
NodeName=n[141-184] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6148

# hendon
NodeName=n[185-186] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6148
NodeName=n[187-188] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6148

# gc3f
NodeName=n[189-192] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6148
NodeName=n[193-196] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6148

# kuhl
NodeName=n[197-200] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6248

# kern
NodeName=n[201-210] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6148
NodeName=n[211-212] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6248
NodeName=n[244] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6248,v100 Gres=gpu:tesla_v100-sxm2-32gb:4
NodeName=n[245-259] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6248
NodeName=n[271-273] Sockets=2 CoresPerSocket=32 ThreadsPerCore=1 RealMemory=1003520 Feature=epyc,a100 Gres=gpu:nvidia_a100-pcie-40gb:3

# amt, melgar
NodeName=n[213-216] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6248
NodeName=n[233-240] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6230

# phillips
NodeName=n[217-218] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=754000 Feature=skylake,6148
NodeName=n[219] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6248

# ctn
NodeName=n[222] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6248

# bgmp
NodeName=n[225-231] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6230
NodeName=n[278-280] Sockets=2 CoresPerSocket=24 ThreadsPerCore=1 RealMemory=247635 Feature=epyc

# cisds
NodeName=n[241-243] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6248,v100 Gres=gpu:tesla_v100-sxm2-32gb:4

# dsci
NodeName=n[261-263] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=180771 Feature=skylake,6230

# karlstrom
NodeName=n[265-266] Sockets=2 CoresPerSocket=20 ThreadsPerCore=1 RealMemory=374307 Feature=skylake,6148

# datascience
NodeName=n[270] Sockets=2 CoresPerSocket=28 ThreadsPerCore=1 RealMemory=6059427 Feature=8280l,optane

# dasa
NodeName=n[274] Sockets=2 CoresPerSocket=24 ThreadsPerCore=1 RealMemory=1020522 Feature=epyc

# preempt
# all nodes sinfo -o "%12P %8D %8c %10m %50f"|grep -v preempt
PARTITION    NODES    CPUS     MEMORY     AVAIL_FEATURES
compute      42       128      504433     amd,milan,7713
compute_inte 84       28       117595     intel,broadwell,e5-2690
computelong  29       128      504433     amd,milan,7713
computelong_ 61       28       117595     intel,broadwell,e5-2690
gpu          5        48       504433     amd,milan,7413,a100,gpu-10gb
gpu          2        48       504433     amd,milan,7413,a100,gpu-80gb,3xgpu-80gb,no-mig
gpu          4        48       504433     amd,milan,7413,a100,gpu-80gb,2xgpu-80gb,no-mig
gpu          8        48       246385     amd,milan,7413,a100,gpu-40gb
gpu          4        48       246385     amd,milan,7413,a100,gpu-80gb,no-mig
gpulong      3        48       504433     amd,milan,7413,a100,gpu-80gb,2xgpu-80gb,no-mig
gpulong      6        48       246385     amd,milan,7413,a100,gpu-40gb
gpulong      2        48       504433     amd,milan,7413,a100,gpu-10gb
gpulong      1        48       504433     amd,milan,7413,a100,gpu-80gb,3xgpu-80gb,no-mig
gpulong      2        48       246385     amd,milan,7413,a100,gpu-80gb,no-mig
interactive  1        48       246385     intel,icelake,6342
interactive  4        28       117595+    intel,broadwell,e5-2690
interactive  13       28       246385     intel,broadwell,e5-2690v4
interactiveg 1        48       504433     amd,milan,7413,a100,gpu-10gb
interactiveg 1        64       504416     amd,rome,7542,a100,gpu-40gb,gpu-80gb,no-mig
memory       1        56       1020522    intel,broadwell,e7-4830,mem-1t
memory       2        56       4117194    intel,broadwell,e7-4830,mem-4tb
memory       4        56       2052811    intel,broadwell,e7-4830,mem-2tb
memory       2        56       4067951    intel,icelake,6348,mem-4tb
memory       4        56       2027750    intel,icelake,6348,mem-2tb
memory       2        56       1003878    intel,icelake,6348,mem-1tb
memory       1        56       1020522    intel,broadwell,e7-4830,mem-1tb
memorylong   2        56       2027750    intel,icelake,6348,mem-2tb
memorylong   1        56       1003878    intel,icelake,6348,mem-1tb
memorylong   1        56       1020522    intel,broadwell,e7-4830,mem-1t
memorylong   1        56       4117194    intel,broadwell,e7-4830,mem-4tb
memorylong   2        56       2052811    intel,broadwell,e7-4830,mem-2tb
memorylong   1        56       4067951    intel,icelake,6348,mem-4tb
amt          2        40       180771     intel,cascadelake,6248
amt          8        40       374307     intel,cascadelake,6230,tmp-9tb
bgmp         3        48       247635     amd,milan,7413
bgmp         7        40       180771     intel,cascadelake,6230
cisds        2        48       1020522    intel,sapphirerapids,6442y,mem-1tb,h100,gpu-80gb
cisds        3        40       374307     intel,cascadelake,6248,v100,gpu-32gb
cryoc        1        64       247635     intel,icelake,6338n
ctn          1        40       180771     intel,cascadelake,6248
dasa         1        48       1020522    amd,milan,7413,mem-1tb
datascience  3        40       180771     intel,cascadelake,6230
datascience  4        48       247635     amd,milan,7413
datascience  1        56       6059427    intel,cascadelake,8280l,mem-6tb
dsci100      1        128      504433     amd,milan,7713
dufek        35       40       374307     intel,skylake,6148
gc3f         8        40       180771+    intel,skylake,6148
hendon       4        40       180771     intel,skylake,6148
jhp          8        28       246385     intel,broadwell,e5-2690
karlstrom    1        40       374307     intel,cascadelake,6248
karlstrom    1        40       374307     intel,skylake,6148
kern         17       40       180771     intel,cascadelake,6248
kern         10       40       374307     intel,skylake,6148
kern         7        128      504433     amd,milan,7713
kern         4        64       504416     amd,milan,7513
kerngpu      2        64       1003520    amd,milan,7543,a100,gpu-20gb
kerngpu      1        64       1003520    amd,milan,7543,a100,gpu-40gb,no-mig
kerngpu      1        40       374307     intel,cascadelake,6248,v100,gpu-32gb
kerngpu      1        112      2052811    intel,sapphirerapids,8480cl,mem-2tb,h100,gpu-80gb
kuhl         4        40       180771     intel,cascadelake,6248
lowd         1        128      1003520    amd,milan,7713,a100,gpu-80gb,no-mig
melgar       2        40       180771     intel,cascadelake,6248
murray       1        32       247635     amd,genoa,9354p,l40,gpu-48gb
paty         8        40       374307     intel,skylake,6148
phillips     1        40       374307     intel,cascadelake,6248
phillips     2        40       754000     intel,skylake,6148
racs         1        40       180771     intel,skylake,6148
racsgpu      1        64       504416     amd,rome,7542,a100,gpu-40gb,gpu-80gb,no-mig
rohlfs       2        56       1003878    intel,icelake,6348,mem-1tb
rohlfs       2        48       247635     intel,icelake,6342

Notes:

  • RealMemory is in MB.

  • sinfo shows only the partitions you have access to, include the flag -a to display information about all partitions.

  • Nodes with a desired Feature may be requested using the srun or sbatch flag --constraint=<list>, i.e. --constraint=v100

  • Hostnames change, avoid specifying a specific list of hosts

  • See slurm informational tools: man sinfo and RACS provided wrapper tools for Slurm information:

Code Block
 /packages/racs/bin/slurm-show-cpu-mem

...


 /packages/racs/bin/slurm-show-features

...


 /packages/racs/bin/slurm-show-gpus
Filter by label (Content by label)
cqllabel in ( "nodes" , "partitions" )

...