How to set up High Performance Computing
AHPCC from University of Arkansas
1 General Information
Arkansas High Performance Computing Center (AHPCC, official website) is available for research and instructional use to faculty and students of any Arkansas university and their research collaborators. There is no charge for use of our computing resources.
To use the HPC, an AHPCC account must be requested through Internal Account Request Form. Please see here for more information about AHPPC inventory. ]
2 Connect to HPC
- You can use online dashboard or terminal to have access to AHPCC nodes.
2.1 For terminal
As long as you have an AHPCC account, you can connect to HPC through SSH. For Windows users, you can use PuTTY or Powershell to connect to HPC. For Mac and Linux users, you can use the terminal to connect to HPC. The command is:
ssh [username]@hpc-portal2.hpc.uark.edu
Replace [username] with your username of AHPCC account. Passwords will be required. After you enter your password, you will be connected to HPC and your terminal/Powershell will look like this.
Note: Pinnacle is a new resource at the University of Arkansas in 2019. It consists of 100 Intel based nodes with 20 NVIDIA V100 GPU nodes enabling data science and machine learning and 8 big memory nodes with 768 Gb ram/each for projects requiring a large memory footprint.
2.2 SSH login without password
- Generate a pair of authentication keys in your local machine and do not enter a passphrase using the following codes:
ssh-keygen -t rsa
Please note that make the passphrase empty:
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/[username]/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/[username]/.ssh/id_rsa
Your public key has been saved in /Users/[username]/.ssh/id_rsa.pub
- In your local machine, type in following commands to copy local public key to the hpc server.
scp ~/.ssh/id_rsa.pub [loginname]@hpc-portal2.hpc.uark.edu:/home/[loginname]/.ssh/authorized_keys
- Now you should be able to login the hpc login node without password:
ssh [loginname]@hpc-portal2.hpc.uark.edu
3 Upload or Download Data
There are two ways to download data: (1) dashboard (2) terminal
3.1 Dashboard
One the dashboard page, find Files
tag on the navigation bar:
Files > /karpinski/[username]
You then should be able to upload or download your files from local to server or form server to local.
3.2 Terminal
- To upload data files from local machine to HPC, type in following codes on your local machine:
scp program.c [username]@hpc-portal2.hpc.uark.edu:/home/username/
where program.c
is one example file you want to upload. If you target file is located in Downloads folder, use ~/Downloads/program.c
instead.
To copy an entire folder and its subfolds using SCP, add parameter -r
following scp
for recursive operations (src
is a folder for example):
scp -r src [username]@hpc-portal2.hpc.uark.edu:/home/username/
- To download data files from HPC to local machine, type in following codes on your local machine:
scp -r [username]@hpc-portal2.hpc.uark.edu:/home/username/src ./
4 Jobs submission
4.1 Workflow
There are multiple steps to submit the R file to cluster to run.
- We need to determine the computing nodes we want to use. Please refer to this link for detailed information about HPC equipment. A general ‘job submit’ command is like this:
sbatch -q q06h32c -l walltime=1:00 -l nodes=1:ppn=32 example.sh
The sbatch
command (a slurm command) aims to submit a job that is saved to the job file example.sh
or example.slurm
. The command above submitted the job to the q06h32c queue with a wall-time of 1 minute requesting all 32 cores on 1 node.
- We want to create a job file with
.sh
or.slurm
extension to automatically run the job submission commands aforementioned. Here’s a simple example of a job fileexample.sh
that can tell HPC how to run your R code (#SBATCH
represents all parameters we feed to job submission):
#!/bin/bash
#SBATCH --job-name=mpi
#SBATCH --output=zzz.slurm
#SBATCH --partition comp06
#SBATCH --nodes=2
#SBATCH --tasks-per-node=32
#SBATCH --time=6:00:00
module load gcc/11.2.1 mkl/19.0.5 R/4.2.2
# module load gcc/11.2.1 intel/21.2.0 mkl/21.3.0 R/4.3.0
Rscript HelloWorld/example.R
R relevant modules
Where Line 8 loaded all required modules:
gcc
andmkl
are required for R package installation (Note: To the date,gcc/11.2.1
is the latest version ofgcc
than can compile thecmdstanr
successfully). Please see here for more details.Rscript
is the bash command to execute R file on HPC.HelloWorld/example.R
is the path of your R script.Anything behind the
#SBATCH
are options for the SLURM scheduler. Please see following summary or view it online: