home > computing > HPC Cluster Instructions


Department HPC Cluster Instructions

To submit a job, use the following steps:
  1. Request Access
    • Email help@stat.ncsu.edu stating you wish to have login access to the beowulf cluster and include your Unity ID.
  2. Transfer your programs to cluster.
    • {from mac} $ scp [program] unityid@hpc.stat.ncsu.edu:.
    • {from pc} start winscp and connect to hpc.stat.ncsu.edu with your Unity ID and password
  3. Login to Cluster
    • {from mac} ssh unityid@hpc.stat.ncsu.edu
    • {from pc}start putty and connect to hpc.stat.ncsu.edu with your Unity ID and password
  4. Submit job

    There are 2 ways to submit jobs.

    If you use bwsubmit, it will submit a job that will allow you to use 1 processor core and up to 8GB of RAM. The run time limit is 1 day (24 hours) for these jobs. This is meant for people who need to run many quick scripts. If your code runs for more than 1 day or you need more than 8GB of RAM, you should to use parallel programming and run with bwsubmit_multi. Because the limit is 1 day, you can use 50 cores at a time. There is a seperate machine and queue for 1 processor jobs, so 128 total cores are available all users combined. After 128 cores are in use, jobs will queue up. Job run times may vary significantly because hyper-threading is enabled on this machine to utilize it completely.

    If you use bwsubmit_multi, you can schedule a job using 4, 8, or 16 processor cores each and 8GB of RAM for each processor core. Request multiple processors to get more memory allocation, even if you do not use the cores. Your job will also be limited to the number of processors you request. If you try to use more than you have requested, the performance of your program will suffer greatly. The BLAS/LAPACK libraries will automatically use the number of cores you request.

    Important Notes: If your program uses more than the allocated amount of RAM, or your job runs for more than 1 day with bwsubmit, 10 days with bwsubmit_multi, the scheduler will kill your job. You can have up to 32 cores worth of jobs running at a time (16 max per job). If the cluster fills up completely, it will limit everyone to 16 cores running at a time to try to serve more people. It automatically adjusts every hour. Do not submit jobs under other users names to bypass the per user core limit. This is a violation of NC State policies and both accounts will be heavily restricted or removed from the cluster.

    • $ bwsubmit [program] [filename] OR bwsubmit_multi [processor_cores] [program] [filename]
    • Examples
      • $ bwsubmit sas program.sas OR bwsubmit_multi 4 sas program.sas
      • $ bwsubmit r sim.r OR bwsubmit_multi 8 r sim.r
      • $ bwsubmit exec fortran.exe OR bwsubmit_multi 16 exec fortran.exe
      • $ bwsubmit matlab prog.m OR bwsubmit_multi 4 matlab prog.m
    • WARNING: System is case sensitive.
      Meaning if your program is named Program.Sas it must be submitted as bwsubmit sas Program.Sas
  5. Check if R Packages are installed.
    R-PkgCheck {package_name1 package_name2}
    This will check and see if the R-Packages you need are already installed on the system. If not installed email help@stat.ncsu.edu asking to install package.
  6. Log Off Cluster
    • $ exit
  7. You'll receive an email when your program has finished running
  8. Transfer your results from cluster to your computer just like step 2 above.

Other useful commands: