Difference between revisions of "Basic Usage: CPU Based Resources With Slurm"

From KENET Training
Jump to: navigation, search
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
== Introduction ==
 
−
Slurm [https://slurm.schedmd.com/documentation.html] is a workload manager for clusters, offering both batch and interactive job scheduling.
 
−
It works over a text based interface on the linux terminal.
 
−
 
−
Slurm will provide you with the following to help you make use of the cluster;
 
−
# What resources are available on the cluster.
 
−
# Queuing and allocation of jobs based on specified resources.
 
−
# Job monitoring and status reporting.
 
  
 
== Simple  commands with SLURM ==
 
== Simple  commands with SLURM ==
Line 35: Line 27:
 
== Submitting Your first Job ==
 
== Submitting Your first Job ==
 
==== Create a submission script ====
 
==== Create a submission script ====
−
You require a submission script, which is a plain text file with all the instructions for the command you intend to run:  
+
You require a submission script, which is a plain text file with all the instructions for the command you intend to run.
 +
Create a working directory in your scratch directory:
 +
<code bash>
 +
  cd ~/localscratch/
 +
  mkdir test
 +
</code>
 +
and in this directory we will place the following text content in a file:
 
<code bash>
 
<code bash>
 
   #!/bin/bash
 
   #!/bin/bash
Line 51: Line 49:
 
   module load applications/qespresso/7.3.1  
 
   module load applications/qespresso/7.3.1  
 
    
 
    
−
   cd $HOME/test
+
   cd $HOME/localscratch/test
 
   mpirun -np 4  pw.x <input.in > output.out
 
   mpirun -np 4  pw.x <input.in > output.out
 
</code>
 
</code>
Line 64: Line 62:
 
</code>
 
</code>
 
This will run the named program on four 4 cores, note that the parallelism is built into the program, if the program itself is not parallelised, running on multiple cores will not provide any benefit.
 
This will run the named program on four 4 cores, note that the parallelism is built into the program, if the program itself is not parallelised, running on multiple cores will not provide any benefit.
 +
 +
Next:
 +
[[Basic_Usage:_GPU_Based_Resources_With_Slurm| Basic_Usage:_GPU_Based_Resources_With_Slurm]]
 +
 +
Up:
 +
[[ HPC_Usage| HPC_Usage]]

Latest revision as of 15:28, 1 April 2025

Simple commands with SLURM

You can obtain information on the Slurm "Partitions" that accept jobs using the sinfo command 

   $ sinfo
   PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
   test         up       1:00      1   idle gnt-usiu-gpu-00.kenet.or.ke
   gpu1         up 1-00:00:00      1   idle gnt-usiu-gpu-00.kenet.or.ke
   normal*      up 1-00:00:00      1   idle gnt-usiu-gpu-00.kenet.or.ke


The test partition is reserved for testing, with a very short time limit. The normal partition is to be used for CPU only jobs, and the gpu1 queue is reserved for GPU jobs. Both production partitions have a time limit of 24 hours at a time for individual jobs.

Showing The Queue

The squeue slurm command will list all submitted jobs, and will give you an indication of how busy the cluster is, as well as the status of all running or waiting jobs. Jobs that are complete will exit the queue and will not be in this list. 

   $ squeue 
   JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
    63    normal     gpu1   jotuya  R       0:03      1 gnt-usiu-gpu-00.kenet.or.ke
   $

Submitting Your first Job

Create a submission script

You require a submission script, which is a plain text file with all the instructions for the command you intend to run. Create a working directory in your scratch directory: 

 cd ~/localscratch/
 mkdir test

and in this directory we will place the following text content in a file: 

 #!/bin/bash

 #SBATCH -J testjob               # Job name
 #SBATCH -o job.%j.out         # Name of stdout output file (%j expands to jobId)
 #SBATCH -e %j.err             # Name of std err
 #SBATCH --partition=normal    # Queue
 #SBATCH --nodes=1             # Total number of nodes requested
 #SBATCH --gres=gpu:1             # Total number of gpus requested
 #SBATCH --cpus-per-task=1     # 
 #SBATCH --time=00:03:00        # Run time (hh:mm:ss) - 1.5 hours
  
 # Launch MPI-based executable
 module load applications/qespresso/7.3.1 
 
 cd $HOME/localscratch/test
 mpirun -np 4  pw.x <input.in > output.out

Put this in a file called *test.slurm*

Submitting the Job to the Queue

The slurm sbatch command provides the means to submit batch jobs to the queue: 

   $ sbatch  test.slurm 
   Submitted batch job 64
   $

This will run the named program on four 4 cores, note that the parallelism is built into the program, if the program itself is not parallelised, running on multiple cores will not provide any benefit.

Next: Basic_Usage:_GPU_Based_Resources_With_Slurm

Up: HPC_Usage

Retrieved from "http://training.kenet.or.ke/index.php?title=Basic_Usage:_CPU_Based_Resources_With_Slurm&oldid=1218"