Sources & References

• Official Ububtu Tutorials
• Linux Shell Commands
• Cheat Sheet by Tahul Saigal
• Mac Homebrew

Remote Connection Environment

Official Instructions

File transfer

File transfer scp -r username@hpc-class.its.iastate.edu:mydir ./mydir

• Pronto’s hpc-class:prontodtn.las.iastate.edu
• Condo’s hpc-class:condodtn.its.iastate.edu

Editing using terminal editor

Pronto

How to use Pronto Use Julia GPU docker containers with singularity

srun --time=01:00:00 --cpus-per-task=1 --partition=interactive --pty /usr/bin/bash

condo2017

Python Environment

• Q&A: https://www.hpc.iastate.edu/faq#python-package

  module avail pip
  module load py-pip/9.0.1-py3-dpds55c
  pip install --user <packagename>
  

Julia environment

• Find the version to download on https://julialang.org/downloads/
• use ‘tar xvzf your_tar_file’ to unpack *.tar.gz
• cd to the unpacked directory, and run ‘./configure –prefix=/your_work_directory’ (software will be installed in this directory)
• issue ‘make’ to compile software

• issue ‘make install’ to install compiled code in the directory specified in the –prefix option on the configure command

• Invoke julia executable by using its full path: /bin/julia (https://julialang.org/downloads/platform/)

  wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.5-linux-x86_64.tar.gz 
  tar –tzf documents.tar.gz # https://phoenixnap.com/kb/extract-tar-gz-files-linux-command-line
   <Julia directory>/bin/julia
  

R environment

1. Load R module: $ module load r
2. Enter R environment: $ R
3. Exit: q()

Create job files using SLURM workload manager

• Introduction: https://www.hpc.iastate.edu/guides/condo-2017/managing-jobs-using-slurm-workload-manager
• Job file generator: https://www.hpc.iastate.edu/guides/condo-2017/slurm-job-script-generator-for-condo
• Steps:
  1. Create a sbatch job file jobname.sh using either texteditor locally or vim, change the postfix to sh manually
  2. In the terminal, under the directory where programName.R locates in, type sbatch jobname.sh
  3. Check if your job has been submitted by qstat -u NetID or qstat -a
• The job file look like this:

    #!/bin/bash
  
    # Copy/paste this job script into a text file and submit with the command:
    #    sbatch thefilename
    # job standard output will go to the file slurm-%j.out (where %j is the job ID)
  
    #SBATCH --time=1:00:00   # walltime limit (HH:MM:SS)
    #SBATCH --nodes=1   # number of nodes
    #SBATCH --ntasks-per-node=16   # 16 processor core(s) per node 
  
    # LOAD MODULES, INSERT CODE, AND RUN YOUR PROGRAMS HERE
    module load r
    R CMD BATCH programName.R
  

• Commands for checking job status

  Commands	Results
  qstat -a	All jobs
  qstat -u username	Jobs submitted by user
  qsub	Execute the jobfile
  qdel	Terminate one job
  export LC_ALL="en_US.UTF-8"	Set language

Basic Commands

Change Directory (cd)

List (ls)

File and Directory Management

Short Cuts

Homebrew & Cask

Useful to install and version control softwares.