Repository navigation
Troubleshooting
Kasper Skytte Andersen edited this page Oct 6, 2026
·
6 revisions
Start with the job's slurmctld log line (grep "sjp:") and AdminComment. More detail:
sjp.config.verbose = true.
| reason | check |
|---|---|
no table |
sjpd running? not observe? no disable file? table_path matches state_dir? |
stale |
sjpd hung or slow slurmctld (systemctl status sjpd, sjp.log); or a backup controller |
no room |
expected on a full cluster; your fallback applies, or set [policy] always_place
|
error |
Lua error just above the sjp line in the slurmctld log |
Other reasons are jobs sjp skips by design (Using sjp in job_submit.lua).
scontrol reconfigure.
The verbose line says why (several tasks without -N 1, no memory request, only one node has room…, about equally good, pinning is off).
Higher-priority jobs took the node within 60 s. Each release is logged with the pending reason.
-
squeue -h -t PD -o "%r" | sort | uniq -c: onlyQOSMaxCpuPerUserLimitandMaxCpuPerAccountcount. -
enforce+[limits] modeset, no disable file. -
idle_fractionin/run/sjp/status.json≥min_idle_share. - Jobs fit one node's free space.
-
global: ~75 s before a pulse, 5 min between pulses. -
flex: users without access are named in the log.
Non-root users can't write /var/log/sjp: use --state-dir (or --log-file, --text-log).