Slurm jobs submitted to guest partitions, using
#SBATCH --account owner-guest and
#SBATCH --partition cluster-guest substituting the proper cluster name, are eligible for preemption by jobs submitted by the group that owns the nodes. To
help minimize the chance of preemption and avoid wasting time and other resources,
nodes owned by groups with (historically) low utilization can be targeted directly
in batch scripts and interactive submissions. It is important to note that any constraint
suggestions are based solely on historical usage information and are not indicative
of the future behavior of any groups.
Suggestions by cluster
Information about owner utilization is presented as a heatmap generated from Slurm logs. Lighter colors mean fewer nodes with the given node feature were in use by owner groups (which is beneficial for guest jobs) while darker colors mean more of the nodes were being used by the owners. The size of the pool of nodes must also be considered when selecting constraints; if an owner group has many nodes available and utilizes only some of them, the remainder will still be available for guest jobs. Selecting constraints for both effective size and owner utilization, then, can help further reduce the likelihood of preemption.
Multiple constraints can be specified at once with logical operators in Slurm directives.
This allows for submission to nodes owned by one of several owner groups at a time
(which might help reduce queue times and increase the number of nodes available) as
well as the specification of exact core counts and available memory. To select from
multiple owner groups' nodes, use the "or" operator; a directive like
#SBATCH -C "group1|group2|group3" will select from nodes in any of the constraints listed. By contrast, the "and" operator can be used to achieve
further specificity in requests. To request nodes owned by a group and with only some
amount of memory, for example, a directive like
#SBATCH -C "group1&m256" could be used. (This will only work where multiple node features are associated with
the nodes and the combination is valid. To view the available node features, the
si2documented on the Slurm page are helpful.)
If the images are not updating, it may be because your browser is caching older versions. Try an uncached reload of the page.