SLURM - Revision history

GiulioFontana: /* SLURM Quality of Service (QOS) */

2026-05-04T14:34:11Z

SLURM Quality of Service (QOS)

← Older revision		Revision as of 14:34, 4 May 2026
Line 84:		Line 84:
	* a maximum of 64 GB of RAM		* a maximum of 64 GB of RAM
	* this access to GPUs:		* this access to GPUs:
	** ~~maximum~~ 1 GPU of type ''gpu:3g.20gb''		** max 1 GPU of type ''gpu:3g.20gb''
	** no GPUs of type ''gpu:40gb=0''		** no GPUs of type ''gpu:40gb=0''
	** no GPUs of type ''gpu:4g.20gb''		** no GPUs of type ''gpu:4g.20gb''

GiulioFontana: /* Default values */

2026-05-04T14:33:35Z

Default values

← Older revision		Revision as of 14:33, 4 May 2026
Line 369:		Line 369:
	== Default values ==		== Default values ==

	The features of SLURM partitions can be inspected with		The features of SLURM partitions, including the '''default values''' which are applied to jobs that do not make explicit requests, can be inspected with

	<pre style="color: lightgrey; background: black;">		<pre style="color: lightgrey; background: black;">
Line 381:		Line 381:
	AllowGroups=ALL AllowAccounts=ALL AllowQos=nogpu,gpulight,gpu,gpuwide,gpuheavy-20,gpuheavy-40		AllowGroups=ALL AllowAccounts=ALL AllowQos=nogpu,gpulight,gpu,gpuwide,gpuheavy-20,gpuheavy-40
	AllocNodes=ALL Default=YES QoS=N/A		AllocNodes=ALL Default=YES QoS=N/A
	-> DefaultTime=01:00:00 DisableRootJobs=NO ExclusiveUser=NO ExclusiveTopo=NO GraceTime=0 Hidden=NO		=> DefaultTime=01:00:00 DisableRootJobs=NO ExclusiveUser=NO ExclusiveTopo=NO GraceTime=0 Hidden=NO
	MaxNodes=UNLIMITED MaxTime=3-00:00:00 MinNodes=0 LLN=NO MaxCPUsPerNode=UNLIMITED MaxCPUsPerSocket=UNLIMITED		MaxNodes=UNLIMITED MaxTime=3-00:00:00 MinNodes=0 LLN=NO MaxCPUsPerNode=UNLIMITED MaxCPUsPerSocket=UNLIMITED
	Nodes=gn01		Nodes=gn01
Line 388:		Line 388:
	State=UP TotalCPUs=48 TotalNodes=1 SelectTypeParameters=NONE		State=UP TotalCPUs=48 TotalNodes=1 SelectTypeParameters=NONE
	JobDefaults=(null)		JobDefaults=(null)
	-> DefMemPerNode=4096 MaxMemPerNode=UNLIMITED		=> DefMemPerNode=4096 MaxMemPerNode=UNLIMITED
	TRES=cpu=48,mem=1011435M,node=1,billing=49,gres/gpu=13,gres/gpu:3g.20gb=5,gres/gpu:40gb=3,gres/gpu:4g.20gb=5		TRES=cpu=48,mem=1011435M,node=1,billing=49,gres/gpu=13,gres/gpu:3g.20gb=5,gres/gpu:40gb=3,gres/gpu:4g.20gb=5
	TRESBillingWeights=cpu=1.0,gres/gpu:3g.20gb=6.0,gres/gpu:4g.20gb=6.0,gres/gpu:40gb=6.0,mem=0.05g		TRESBillingWeights=cpu=1.0,gres/gpu:3g.20gb=6.0,gres/gpu:4g.20gb=6.0,gres/gpu:40gb=6.0,mem=0.05g
	</pre>		</pre>

	In the example, we have highlighted with "->" the most relevant for Mufasa users, i.e. ~~two '''default values''' which are applied to jobs that do not make explicit requests. Precisely~~:		In the example, we have highlighted with "=>" the most relevant default values for Mufasa users, i.e.:

	;<code>DefaultTime</code>		;<code>DefaultTime</code>

GiulioFontana: /* Partition availability */

2026-05-04T14:32:06Z

Partition availability

← Older revision		Revision as of 14:32, 4 May 2026
Line 363:		Line 363:
	:: Queued jobs will be executed when the partition becomes available again (i.e. goes back to the <code>up</code> state)		:: Queued jobs will be executed when the partition becomes available again (i.e. goes back to the <code>up</code> state)

	When a partition goes from <code>up</code> to <code>drain</code> no harm is done to running jobs. ~~When~~ a ~~partition passes~~ from ~~any other state~~ to <code>down</code>, running ~~jobs (if~~ they ~~exist)~~ get killed. A partition in state <code>drain</code> or <code>down</code> requires intervention by a [[Roles\|Job Administrator]] to be restored to <code>up</code>.		When a partition goes from <code>up</code> to <code>drain</code> no harm is done to running jobs. In a normally functioning SLURM system, the passage from <code>up</code> or <code>drain</code> to <code>down</code> happens only when no jobs are running on the partition. If (e.g., due to a malfunction) the passage happens with jobs still running, they get killed.

			A partition in state <code>drain</code> or <code>down</code> requires intervention by a [[Roles\|Job Administrator]] to be restored to <code>up</code>.

	== Default values ==		== Default values ==

GiulioFontana: /* Partition availability */

2026-05-04T14:28:02Z

Partition availability

← Older revision		Revision as of 14:28, 4 May 2026
Line 351:		Line 351:
	:: Currently running jobs will be completed		:: Currently running jobs will be completed
	:: It's possible to launch jobs on the partition		:: It's possible to launch jobs on the partition
	:: ~~Currently queued~~ jobs will be executed as soon as resources allow		:: Queued jobs will be executed as soon as resources allow

	:'''<code>drain</code>''' = the partition is in the process of becoming unavailable (i.e., of entering the <code>down</code> state: see below)		:'''<code>drain</code>''' = the partition is in the process of becoming unavailable (i.e., of entering the <code>down</code> state: see below)

GiulioFontana: /* Partition availability */

2026-05-04T14:27:25Z

Partition availability

← Older revision		Revision as of 14:27, 4 May 2026
Line 349:		Line 349:

	:'''<code>up</code>''' = the partition is available		:'''<code>up</code>''' = the partition is available
			:: Currently running jobs will be completed
	:: It's possible to launch jobs on the partition		:: It's possible to launch jobs on the partition
	~~:: Currently running jobs will be completed~~
	:: Currently queued jobs will be executed as soon as resources allow		:: Currently queued jobs will be executed as soon as resources allow

	:'''<code>drain</code>''' = the partition is in the process of becoming unavailable (i.e., of entering the <code>down</code> state)		:'''<code>drain</code>''' = the partition is in the process of becoming unavailable (i.e., of entering the <code>down</code> state: see below)
			:: Currently running jobs will be completed
	:: It's not possible to launch jobs on the partition		:: It's not possible to launch jobs on the partition
	~~:: Currently running jobs will be completed~~
	:: Queued jobs will be executed when the partition becomes available again (i.e. goes back to the <code>up</code> state)		:: Queued jobs will be executed when the partition becomes available again (i.e. goes back to the <code>up</code> state)

	:'''<code>down</code>''' = the partition is unavailable		:'''<code>down</code>''' = the partition is unavailable
			:: There are no running jobs
	:: It's not possible to launch jobs on the partition		:: It's not possible to launch jobs on the partition
	~~:: There are no running jobs~~
	:: Queued jobs will be executed when the partition becomes available again (i.e. goes back to the <code>up</code> state)		:: Queued jobs will be executed when the partition becomes available again (i.e. goes back to the <code>up</code> state)

GiulioFontana: /* Partition availability */

2026-05-04T14:26:14Z

Partition availability

← Older revision		Revision as of 14:26, 4 May 2026
Line 346:		Line 346:
	== Partition availability ==		== Partition availability ==

	The most important information that <code>sinfo</code> provides is the '''availability''' (also called ''state'') of partitions. This is shown in column "AVAIL". Possible partition states are:		The most important information that <code>sinfo</code> provides is the '''availability''' (also called '''state''') of partitions. This is shown in column "AVAIL". Possible partition states are:

	:'''<code>up</code>''' = the partition is available		:'''<code>up</code>''' = the partition is available

GiulioFontana: /* SLURM partitions */

2026-05-04T14:25:20Z

SLURM partitions

← Older revision		Revision as of 14:25, 4 May 2026
Line 343:		Line 343:

	Command <code>sinfo</code> does not tell you about the ''jobs'' submitted to a partition. This information is obtained, instead, with [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].		Command <code>sinfo</code> does not tell you about the ''jobs'' submitted to a partition. This information is obtained, instead, with [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].


	== Partition availability ==		== Partition availability ==

GiulioFontana: /* SLURM partitions */

2026-05-04T14:25:08Z

SLURM partitions

← Older revision		Revision as of 14:25, 4 May 2026
Line 342:		Line 342:
	The asterisk at the end of the partition name indicates the default partition, i.e. the one on which jobs which do not ask for a specific partition are run.		The asterisk at the end of the partition name indicates the default partition, i.e. the one on which jobs which do not ask for a specific partition are run.

	Command <code>sinfo</code> does not tell you about the ''jobs'' submitted to a partition. This information ~~can be~~ obtained ~~via~~ [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].		Command <code>sinfo</code> does not tell you about the ''jobs'' submitted to a partition. This information is obtained, instead, with [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].

GiulioFontana: /* SLURM partitions */

2026-05-04T14:24:19Z

SLURM partitions

← Older revision		Revision as of 14:24, 4 May 2026
Line 342:		Line 342:
	The asterisk at the end of the partition name indicates the default partition, i.e. the one on which jobs which do not ask for a specific partition are run.		The asterisk at the end of the partition name indicates the default partition, i.e. the one on which jobs which do not ask for a specific partition are run.

	~~Note that to inspect~~ the ''jobs'' submitted to partition ~~<code>jobs</code> you should use~~ [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].		Command <code>sinfo</code> does not tell you about the ''jobs'' submitted to a partition. This information can be obtained via [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].

GiulioFontana: /* SLURM partitions */

2026-05-04T14:23:03Z

SLURM partitions

← Older revision		Revision as of 14:23, 4 May 2026
Line 341:		Line 341:

	The asterisk at the end of the partition name indicates the default partition, i.e. the one on which jobs which do not ask for a specific partition are run.		The asterisk at the end of the partition name indicates the default partition, i.e. the one on which jobs which do not ask for a specific partition are run.

			Note that to inspect the ''jobs'' submitted to partition <code>jobs</code> you should use [[User Jobs#Inspecting jobs with squeue\|command <code>squeue</code>]].


	== Partition availability ==		== Partition availability ==