This guide provides the basics of connecting to and using the Talapas HPC cluster.

Getting a Talapas Account

If you don't already have an account, see the Request Access page to get one.

Logging on to Talapas

For UO users, your username on Talapas will be your Duck ID.  (That is, if your email address is alice@uoregon.edu, your Talapas username will be "alice".)  Your password is the same university-wide, and can be managed at the UO the UO password reset page.

For non-UO users, you will have received a username and password in the email granting you access.

Talapas currently has two four login nodes:

• login1.talapas.uoregon.edu
• login2.talapas.uoregon.edu
• login3.talapas-ln1.uoregon.edu
• login4.talapas-ln2.uoregon.edu

These hosts are entirely equivalent.

  You can use whichever seems less busy, or use hostname talapas-login  but the recommended method is by logging in via the load balancer:

         login.talapas.uoregon.edu

to be sent to one randomlylogged into one of the four nodes with the least amount of traffic.

If you are logging in from a Linux or Mac OS X workstation, open a terminal and type

ssh myusername@talapas-ln1myusername@login.talapas.uoregon.edu

If you are logging in from Windows, download an SSH client like Putty or MobaXterm and do the equivalent.

...

If you're accessing Talapas from Linux or Mac OS X, you can use scp or rsync to  to transfer files.  For example, type

scp chr1.fasta myusername@talapas-ln1myusername@login[1-4].uoregon.edu:.

to copy the named file to your Talapas home directory in either of the 4 login nodes (login1,...,login4)

There are also GUI tools available for file transfer via SCP or SFTP.  Filezilla is available for all common platforms.

Also,  /wiki/spaces/HPC/pages/1769536 Globus is available, and is particularly useful for the transfer of large files.

...

Here's a link to our wiki page, How-to: Use Globus

Storing your Data

As a Talapas user, you have these directories available for storing and manipulating your files.

...

See Storage for more details.

Software

Read about how to find software on our Software page.  Read on for the absolute basics.

Talapas uses the Lmod environment module software to provide access to the various installed software packages.  Type

...

See How-to Use LMOD for more details.

Running a Batch Job

Talapas uses the SLURM job scheduler and resource manager to provide a way to submit large computational tasks to the cluster in a batch fashion.

As a basic example, here is a simple batch script to run the hostname command on one compute node.  To run it, first create a file hostname.batch containing these lines:

...

...

SLURM Accounts

In order to ensure correct service charges, all SLURM jobs on Talapas require that an account be specified.  You can do this explicitly using the --account (or -A) option to sbatch or srun.  This is shown in the first example below, but omitted from the rest for clarity.  Alternatively, you can set the environment variables SLURM_ACCOUNT and SBATCH_ACCOUNT in your ~/.bash_profile if you like.

If your SLURM job invokes subsidiary calls to 'sbatch', you might also need to include lines like this in your batch script to make sure that the subsidiary 'sbatch' calls also see the account information:

export SLURM_ACCOUNT=$SLURM_JOB_ACCOUNT
export SBATCH_ACCOUNT=$SLURM_JOB_ACCOUNT

Note that if you a member of multiple PIRGs, it is your responsibility to specify the correct account for each job.

If you're not sure what your PIRG/account name is, you can use the groups command.  It will show a list of groups that you're in.  A few are system groups that you cannot charge to (e.g., talapas, gaussian).  But within that list you should recognize the PIRGs that you are a member of.

Running a Batch Job

Talapas uses the SLURM job scheduler and resource manager to provide a way to submit large computational tasks to the cluster in a batch fashion.

As a basic example, here is a simple batch script to run the hostname command on one compute node.  To run it, first create a file hostname.batch containing these lines:

#!/bin/bash
#SBATCH --account=<myaccount>   ### change this to your actual account for charging
#SBATCH --partition=short       ### queue to submit to
   ### number of nodes to use#SBATCH --job-name=myhostjob    ### job name
#SBATCH --ntasks-per-node=1  output=hostname.out   ### numberfile ofin taskswhich to launchstore perjob nodestdout
#SBATCH --cpus-per-task=1error=hostname.err    ### file in which to store job stderr
#SBATCH --time=5           ### number of cores for each### taskwall-clock time hostname

Then, to actually submit the job for execution, type

sbatch hostname.batch

When a job is submitted, SLURM considers all of the resources requested (e.g., CPU cores, memory, time, GPUs) together with the set of currently running and queued jobs.  Once compute nodes are available to run your job, it will be run.

Note that your job will not run until the resources you have requested are available.  This means that you may be able to get your job to run sooner by requesting fewer resources.  For example, if your job will run in 1000MB, it's better not to request 10000MB, as it may take significantly longer for those resources to become available.

Batch jobs are run in a manner that's not dependent on your current terminal, so once a job is submitted, it no longer depends on your being logged in, etc.  Job output will be available in the specified files when it has completed.  You can monitor your job's progress using the squeue and sacct commands.  For example

$ sbatch hostname.batch
Submitted batch job 12345

# checking with squeue, we see that the job has started running (state is 'R')
$ squeue -j 12345
limit, in minutes
#SBATCH --mem=100M              ### memory limit per node, in MB
#SBATCH --nodes=1               ### number of nodes to use
#SBATCH --ntasks-per-node=1     ### number of tasks to launch per node
#SBATCH --cpus-per-task=1       ### number of cores for each task

hostname

Then, to actually submit the job for execution, type

sbatch hostname.batch

When a job is submitted, SLURM considers all of the resources requested (e.g., CPU cores, memory, time, GPUs) together with the set of currently running and queued jobs.  Once compute nodes are available to run your job, it will be run.

Note that your job will not run until the resources you have requested are available.  This means that you may be able to get your job to run sooner by requesting fewer resources.  For example, if your job will run in 1000MB, it's better not to request 10000MB, as it may take significantly longer for those resources to become available.

Batch jobs are run in a manner that's not dependent on your current terminal, so once a job is submitted, it no longer depends on your being logged in, etc.  Job output will be available in the specified files when it has completed.  You can monitor your job's progress using the squeue and sacct commands.  For example

$ sbatch hostname.batch
Submitted batch job 12345

# checking with squeue, we see that the job has started running (state is 'R')
$ squeue -j 12345
   JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
    12345   short     myhostjob   joec  R       5:08      1 n003

# later, we see that the job has successfully completed
$ sacct -j 12345
       JobID    JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
12345       myhostjob+      short    joegrp         1     COMPLETED      0:0
12345.batch      batch               joegrp         1     COMPLETED      0:0
12345.exte+     extern               joegrp         1     COMPLETED      0:0

...

srun --account=hpcrcf --pty --partition=short --mem=1024M --time=240 bash

...

The above command is good for running single programs.  If you'll be doing something that will invoke multiple parallel processes or threads, like a parallel make or multi-threaded program, you might want to add a --cpus-per-task=N flag to allocate more cores.

...

  Note that on Talapas, using --ntasks (or -n) instead of --cpus-per-task (or -c) is virtually always the wrong thing to do, and will make your session run more slowly.

Running Graphical Interactive Jobs

If your interactive job will involve a program like rstudio, which uses X11 to provide graphical output, there are several options.  You can use the srun --x11 flag like so to forward your X11 connection back to your workstation X server.

srun --account=hpcrcf --x11 --pty --partition=short --mem=4G --time=4:00 bash

...

xrun --account=hpcrcf --partition=short --mem=1024 --time=4 bash

...

sacct -j 301111 --format=JobID,JobName,ReqMem,MaxRSS,Elapsed
       JobID    JobName     ReqMem     MaxRSS    Elapsed
------------ ---------- ---------- ---------- ----------
301111        myjobname     3800Mc              16:00:28
301111.batch      batch     3800Mc    197180K   16:00:30
301111.exte+     extern     3800Mc     3800Mc      2172K   16:00:29

This job used about 193MB (197180K) of RAM at its peak, and ran for a bit over 16 hours.

...

 2172K   16:00:29

This job used about 193MB (197180K) of RAM at its peak and ran for a bit over 16 hours.

There's also a helper script 'seff' that provides similar information and is a bit easier to type.  Just say "seff JOBID".

See Memory for more details.

Special Handling for Quarterly Maintenance Events

Approximately once per quarter, Talapas is taken down for maintenance, to update software and hardware, make repairs, etc.  To minimize the disruption, SLURM reservations are added to "block out" those times.  If you take no special steps, a job request that you make that runs into a maintenance window will be held and run after the window.

This is often what you'd want.  But, sometimes you might want a job to start right away, even if the entire optimal running time is not available.  You can specify this using the "--min-time" flag to specify a minimum acceptable time limit for the job.

...
#SBATCH --time=0-8:00:00
#SBATCH --time-min=0-1:00:00
...

Using the above flag, you'll normally get a time limit of eight hours.  But, if there are reservations that would delay the start of your job, SLURM will consider any  time limit down to one hour to be acceptable and start the job immediately.  (SLURM will prefer the largest available time limit.)

Being a Good Cluster Citizen

...