-
Notifications
You must be signed in to change notification settings - Fork 26
/
RELEASE_NOTES
189 lines (174 loc) · 8.28 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
RELEASE NOTES FOR SLURM VERSION 2.1
8 March 2010
IMPORTANT NOTE:
SLURM state files in version 2.1 are different from those of version 2.0.
After installing SLURM version 2.1, plan to restart without preserving
jobs or other state information. While SLURM version 2.0 is still running,
cancel all pending and running jobs (e.g.
"scancel --state=pending; scancel --state=running"). Then stop and restart
daemons with the "-c" option or use "/etc/init.d/slurm startclean".
If using the slurmdbd (SLURM DataBase Daemon) you must update this first.
The 2.1 slurmdbd will work with SLURM daemons at version 2.0.0 and above.
You will not need to update all clusters at the same time, but it is very
important to update slurmdbd first and having it running before updating
any other clusters making use of it. No real harm will come from updating
your systems before the slurmdbd, but they will not talk to each other
until you do.
There are substantial changes in the slurm.conf configuration file. It
is recommended that you rebuild your configuration file using the tool
doc/html/configurator.html that comes with the distribution.
HIGHLIGHTS
* The sched/gang plugin has been removed. The logic is now directly within the
slurmctld daemon so that gang scheduling and/or job preemption can be
performed with a backfill scheduler.
* Sbatch response changed from "sbatch: Submitted batch job #" written to
stderr to "Submitted batch job #" written to stdout.
* Preempted jobs can now be canceled, checkpointed or requeued rather than
only suspended.
* Support for QOS (Quality Of Service) has been added to the accounting
database with configurable limits, priority and preemption rules.
* Added "--signal=<int>@<time>" option to salloc, sbatch and srun commands to
notify programs before reaching the end of their time limit.
* Added squeue option "--start" to report expected start time of pending jobs.
The times are only set if the backfill or builtin scheduler is in use.
* The pam_slurm Pluggable Authentication Module for SLURM previously
distributed separately has been moved within the main SLURM distribution
and is packaged as a separate RPM.
* Support has been added for OpenSolaris.
* Added environment variable support to sattach, salloc, sbatch and srun
to permit user control over exit codes so application exit codes can be
distinguished from those generated by SLURM. SLURM_EXIT_ERROR specifies the
exit code when a SLURM error occurs. SLURM_EXIT_IMMEDIATE specifies the
exit code when the --immediate option is specified and resources are not
available. Any other non-zero exit code would be that of the application
run by SLURM.
CONFIGURATION FILE CHANGES (see "man slurm.conf" for details)
* Added PreemptType parameter to specify the plugin used to identify
preemptable jobs (partition priority or quality of service) and
PreemptionMode to identify how to preempt jobs (requeue, cancel, checkpoint,
or suspend). Default is no job preemption.
* The sched/gang plugin has be removed, use PreemptType=preempt/partition_prio
and PreemptMode=suspend,gang.
* ControlMachine changed to accept multiple comma-separated hostnames for
support of some high-availability architectures.
* Added MaxTasksPerNode to control how many tasks that the slurmd can launch.
The default value is 128 (same as Slurm v2.0 value).
* Removed SrunIOTimeout parameter. It now uses MessageTimeout value.
* Added SchedulerParameters option of "max_job_bf=#" to control how far down
the queue of pending jobs that SLURM searches in an attempt backfill
schedule them. The default value is 50 jobs.
COMMAND CHANGES (see man pages for details)
* Added "sacctmgr show problems" command to display problems in the accounting
database (e.g. accounts with no users, users with no UID, etc.).
* Add support for job step time limits.
* Several redundant squeue output and sorting options have been removed:
"%o" (use %D"), "%b" (use "%S"), "%X", %Y, and "%Z" (use "%z").
* Standardized on the use of the '-Q' flag for all commands that offer the
--quiet option.
* salloc's --wait=<secs> option deprecated by --immediate=<secs> option to
match the srun command.
* Scalability of sview dramatically improved.
* Added reservation flag of "OVERLAP" to permit a new reservation to use
nodes already in another reservation.
* Added to sacct the ability to use --format NAME%LENGTH similar to sacctmgr.
* For salloc, sbatch and srun commands:
Ignore _maximum_ values for --sockets-per-node, --cores-per-socket and
--threads-per-core options.
Remove --mincores, --minsockets, --minthreads options (map them to
minimum values of -sockets-per-node, --cores-per-socket and
--threads-per-core for now).
Changed the single character option for dependency from "-P" to the
more intuitive, "-d". This obsoletes the use of srun -d to set a
slurmd debug level. Use srun --slurmd-debug instead.
* Changed "scontrol show job" command:
ReqProcs (number of processors requested) is replaced by NumCPUs
(number of cpus requested or actually allocated)
ReqNodes (number of nodes requested) is replaced by NumNodes
(number of nodes requested or actually allocated).
Added a --detail option to "scontrol show job" to display the
cpu/memory allocation information on a node-by-node basis.
Reorganized the output into functional groupings.
BLUEGENE SPECIFIC CHANGES
* scontrol show blocks option added.
* scontrol delete block and update block can now remove blocks on dynamic
layout configuration.
* sinfo and sview now display correct CPU counts for partitions.
* Jobs waiting for a block to boot will now be reported in Configuring state.
* Vastly improve dynamic layout mode algorithm.
* Environment variables such as SLURM_NNODES, SLURM_JOB_NUM_NODES and
SLURM_JOB_CPUS_PER_NODE now reference cnode counts instead of midplane
counts. SLURM_NODELIST still references midplane names.
API CHANGES
* General changes:
Replaced use of the term "procs" with "cpus"
Eliminated min/max specifications for sockets/cores/threads
* Changed the following struct definitions:
allocation_msg_thread_t
jobacctinfo_t
select_jobinfo_t
slurm_cred_t
switch_jobinfo_t
* Added the following struct definitions:
block_info_msg_t
block_info_t
job_resources_t
job_sbcast_cred_msg
sbcast_cred_t
* Renamed select_job_res_t to select_nodeinfo_t
* Changed members of the following structs:
job_descriptor
job_step_info_t
node_info
node_info_msg
partition_info
reserve_info
resv_desc_msg
slurm_ctl_conf
slurm_step_ctx_params_t
slurm_step_launch_params_t
* Changed members of the job_info struct
Note that cpu_count_reps, cpus_per_node, and num_cpu_groups were moved
to new job_resources struct
* Changed the following enums
job_state_reason
node_states
select_print_mode
* Added the select_nodedata_type enum
* Renamed the select_data_type enum to select_jobdata_type
* Added the following API's
slurm_ctl_conf_2_key_pairs()
slurm_free_block_info_msg()
slurm_free_sbcast_cred_msg()
slurm_get_select_nodeinfo()
slurm_init_update_block_msg()
slurm_job_cpus_allocated_on_node()
slurm_job_cpus_allocated_on_node_id()
slurm_job_node_ready()
slurm_load_block_info()
slurm_print_block_info()
slurm_print_block_info_msg()
slurm_sbcast_lookup()
slurm_sprint_block_info()
* Changed the following API's
slurm_jobinfo_ctx_get()
slurm_load_job()
slurm_load_jobs()
slurm_print_node_table()
slurm_sprint_node_table()
OTHER CHANGES
* A mechanism has been added for SPANK plugins to set environment variables
for Prolog, Epilog, PrologSLurmctld and EpilogSlurmctld programs using the
functions spank_get_job_env, spank_set_job_env, and spank_unset_job_env. See
"man spank" for more information.
* Set a node's power_up/configuring state flag while PrologSlurmctld is
running for a job allocated to that node.
* Added sched/wiki2 (Moab) JOBMODIFY command support for VARIABLELIST option
to set supplemental environment variables for pending batch jobs.
* The RPM previously named "slurm-aix-federation-<version>.rpm" has been
renamed to just "slurm-aix-<version>.rpm" (the federation switch plugin may
not be present).
* Environment variables SLURM_TOPOLOGY_ADDR and SLURM_TOPOLOGY_ADDR_PATTERN
added to describe the network topology for each launched task when
TopologyType=topology/tree is configured
* Add new job wait reason, ReqNodeNotAvail: Required node is not available
(down or drained).