How to set up High Performance Computing of University of Arkansas

Tutorial
HPC
Author

Jihong Zhang

Published

January 14, 2024

1 General Information

Arkansas High Performance Computing Center (AHPCC) is available for research and instructional use to faculty and students of any Arkansas university and their research collaborators. There is no charge for use of our computing resources.

To use the HPC, an AHPCC account must be requested through Internal Account Request Form. Please see here for more information about AHPPC inventory.

2 Connect to HPC

As long as you have an AHPCC account, you can connect to HPC through SSH. For Windows users, you can use PuTTY to connect to HPC. For Mac and Linux users, you can use the terminal to connect to HPC. The command is:

ssh [loginname]@hpc-portal2.hpc.uark.edu

If your account was successfully setted up, passwords will be required. After you enter your password, you will be connected to HPC.

Login Screenshot

Login Screenshot

Note: Pinnacle is a new resource at the University of Arkansas in 2019. It consists of 100 Intel based nodes with 20 NVIDIA V100 GPU nodes enabling data science and machine learning and 8 big memory nodes with 768 Gb ram/each for projects requiring a large memory footprint.

2.1 SSH login without password

  1. Generate a pair of authentication keys in your local machine and do not enter a passphrase using the following codes:
ssh-keygen -t rsa   

Please note that make the passphrase empty:

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/[username]/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/[username]/.ssh/id_rsa
Your public key has been saved in /Users/[username]/.ssh/id_rsa.pub
  1. In your local machine, type in following commands to copy local public key to the hpc server.
scp ~/.ssh/id_rsa.pub [loginname]@hpc-portal2.hpc.uark.edu:/home/[loginname]/.ssh/authorized_keys
  1. Now you should be able to login the hpc login node without password:
ssh [loginname]@hpc-portal2.hpc.uark.edu

3 Moving data

For moving data files from local machine to HPC, type in following codes on your local machine:

scp program.c loginname@hpc-portal2.hpc.uark.edu:/home/loginname/

To copy an entire directory tree src using SCP, use -r for recursive:

scp -r src loginname@hpc-portal2.hpc.uark.edu:/home/loginname/

Similarly, for moving data files from HPC to local machine, type in following codes on your local machine:

scp -r loginname@hpc-portal2.hpc.uark.edu:/home/loginname/src ./

4 Jobs submission

There are multiple steps to submit the R file to cluster to run.

First, we need to determine the computing nodes we want to use. Please refer to this link for detailed information about HPC equipment. A general ‘submitting’ command is like this:

sbatch -q q06h32c -l walltime=1:00 -l nodes=1:ppn=32 example.sh
Note

sbatch command aims to submit a job with the job file example.sh. The command above submitted the job to the q06h32c queue with a wall-time of 1 minute requesting all 32 cores on 1 node.

Second, create a job file with .sh extension. Here’s a simple example of a job file example.sh that can tell HPC how to run your R code:

#!/bin/bash
#SBATCH --job-name=mpi
#SBATCH --output=zzz.slurm
#SBATCH --partition comp06
#SBATCH --nodes=2
#SBATCH --tasks-per-node=32
#SBATCH --time=6:00:00
module load gcc/11.2.1 mkl/19.0.5 R/4.2.2

Rscript HelloWorld/example.R

Note

Where Line 8 loaded all required modules:

  • gcc and mkl are required for R package installation (Note: To the date, gcc/11.2.1 is the latest version of gcc than can compile the cmdstanr successfully). Please see here for more details.

  • Rscript is the bash command to execute R file on HPC. HelloWorld/example.R is the path of your R script.

  • Anything behind the #SBATCH are options for the SLURM scheduler. Please see following summary or view it online: