Skip to content

Plugin Job Manager

Jump to bottom
Hiromichi Yokoyama edited this page Aug 26, 2026 · 7 revisions

Plugin: Job Manager

Job Manager submits calculations to a remote HPC cluster over SSH — or runs them right on this machine, with no SSH at all — tracks their queue status, and fetches the results back into MoleditPy. It is the piece between an input generator writing a file and a result analyzer opening one: upload, submit, watch, fetch, open.
Repository moleditpy_job_manager
Author HiroYokoyama
Supported OS Windows, macOS, Linux, WSL
Supported Python >=3.9, <3.15
Menu location Extensions ▸ Job Manager ▸ Job Monitor / ▸ Submit Job...
Extra dependencies None — uses the system ssh/scp. pip install paramiko only if a host needs password auth

See Official Plugins for the full catalogue.

1. Overview

Job Manager is a queue-agnostic front end: the same submit wizard and job table work whether the target is SLURM, PBS/Torque, SGE/UGE, native Windows (PowerShell), or a machine with no scheduler at all — including this one, with nothing installed. A generated wrapper script installs exit-code and signal traps around your command before it runs, which is how the plugin tells FAILED from LOST from a walltime kill identically across every backend, without depending on sacct or site-specific qstat -f parsing.

2. Adding a host

Hosts… opens a dialog for hostname, user, scheduler, and a remote working directory; Test Connection checks it before you rely on it. Four backends are available:

OpenSSH (default) paramiko (optional) This machine WSL
Install nothing pip install paramiko nothing (bash, or the Windows scheduler) nothing (a WSL distro with bash)
Auth keys / ssh-agent keys, agent and passwords none needed none needed
~/.ssh/config inherited automatically HostName/User/Port/IdentityFile not applicable not applicable
Job chaining the queue's own dependency flag the queue's own dependency flag the wrapper waits for the process the wrapper waits for the process

The OpenSSH backend runs ssh in batch mode, so a background thread never blocks on an invisible password prompt and no password ever reaches the process table. A host that only accepts passwords needs the paramiko backend with Ask for a password when connecting ticked. Passwords are never written to disk — held in memory for the session, asked for again after a restart. Unknown host keys are rejected, not silently trusted.

Running on Windows natively needs nothing installed: the wrapper, the status checks and the plugin's own commands are all PowerShell, which ships with the OS, including a PowerShell-flavoured helper queue for hosts with no scheduler. Running inside WSL from Windows keeps the job directory, the wrapper and the outputs on the Linux side — the same bash script a cluster would run — with input files translated and copied across via wslpath.

3. Submitting a job

New Job… picks the host, takes an input file (or several), and fills in walltime / nodes / memory / modules, with a Template… dropdown carrying a conventional invocation for every program MoleditPy writes input for — ORCA, Gaussian, CP2K, GAMESS, MOPAC, NWChem, Psi4, PySCF, Quantum ESPRESSO, VASP and xTB — with the caveats that matter (ORCA needs its own absolute path to start MPI workers; g16 writes its own .log; VASP takes no input filename at all). A Script preview tab shows the exact wrapper before you submit, and the whole form saves as a named preset.

ORCA, Gaussian, Psi4, NWChem, Q-Chem and GAMESS inputs are read for their own memory and core requests, so the wizard fills those fields in on its own — ORCA's %maxcore is per core and gets multiplied out — without overwriting anything already typed by hand.

Reusing another job's file

A checkbox — Use Job Manager relay tag, exposed in ORCA Input Generator Pro and Gaussian Input Generator Pro directly — writes a [prevfile:.ext] tag into the input in place of a filename: an ORCA * xyzfile, a Gaussian %oldchk, anything a program reads by way of a path. At submit time, Job Manager resolves the tag to the real filename from whichever job you pick as the source (same host only) and copies the file across on the host itself — nothing is downloaded here and re-uploaded. [prevfile:.res/.xyz] reaches one directory down, for a result kept inside a folder of its own (the extension before the slash names the folder, the one after names the file inside it — [prevfile:.res]/[prevfile:.xyz] written as two separate tags does not do the same thing, since each tag then resolves to its own flat, top-level file rather than a folder and the file inside it).

The source job need not have finished yet: picking an active job chains the new submission to start only once it succeeds, and the copy is written directly into the generated wrapper script — run after whatever already gates the job's start — so it is only ever attempted once the file actually exists.

Work that is already on the host

Tick Work already on the host and give a directory instead of uploading anything — ~/runs/mol42 and a command is a whole job. An input file named in that directory is optional; Check confirms it is there before you commit. The directory is checked, never created, and everything the wrapper writes into it carries the job id, so several jobs can share one prepared directory without overwriting each other's results.

Chaining jobs

"Run this after that one" uses each scheduler's own mechanism (--dependency=afterok, -W depend, -hold_jid), or — where there is no queue at all — a wrapper that waits for the previous process. Ask for afterany when the jobs are independent of each other's success. A job stranded behind a failure under a dependency the queue can never satisfy is shown as BLOCKED, not a permanently misleading PENDING.

Batch drag-and-drop

Drop several input files on the Job Monitor and each becomes its own job by default, named after itself and prefilled with a chosen command template. Hold Shift while dropping to keep the older behaviour instead — one job with every dropped file uploaded to it, for a command that genuinely wants more than one input.

4. The helper queue

A host with no scheduler at all — including this machine — still needs jobs serialised sensibly. The helper queue dispatches by physical cores and memory, not by a job count: eight single-core jobs run together on an eight-core box, two 90 GB jobs never share 120 GB, and the rest wait. It needs nothing installed, holds the queue in plain numbered shell scripts readable over ssh, keeps moving with MoleditPy closed, and exits by itself once the queue is empty. A chained-lane mode is available too, for hosts where nothing at all may be left running unattended.

The helper's own runner script is named after the plugin's own version (moleditpy_runner_v<version>.sh / .ps1), not a content hash — a stale copy is superseded by uploading a new one under a new name the next time a job is submitted from a newer plugin version. Old copies are left on the host rather than deleted: a script that ran a job is worth keeping, and the queue is readable over plain ssh precisely so that is possible.

5. Tracking and results

The job table shows queue id, state, elapsed time, and what a job is queued behind; a double click opens the log while a job runs and the result once it has finished. Status survives closing the window, closing the project, and restarting MoleditPy — tracking resumes by itself at launch. Click a column header to sort (by the real elapsed seconds and real timestamp, not the formatted text) and type to filter across every column at once.

State Means
SUBMITTED / PENDING queued
RUNNING / COMPLETING on a node
DONE wrapper finished, exit code 0
FAILED wrapper finished, non-zero — the code is shown in the table
CANCELLED you cancelled it
LOST gone from the queue with no exit code recorded
QUEUED chained behind another job that has not finished yet
BLOCKED behind a job that failed, under a dependency the queue can never satisfy

FAILED (rc=143) is the signature of a job the scheduler itself killed — 143 is 128 + SIGTERM, i.e. walltime exceeded, preemption, or a node drain. LOST means the job left the queue without the wrapper writing its exit code at all — a hard kill, an OOM, a node falling over — and the remote directory is still listed in the tooltip, with Download and Tail Log both still working.

A counter appears in MoleditPy's own status bar while anything is running, queued or blocked, and opens the monitor when clicked; nothing is shown, and nothing is polled, when there are no jobs. A desktop notification names the job and host when one finishes, fails, or vanishes from the queue — on by default, one checkbox away from off.

Chat alerts

A desktop notification reaches whoever is sitting at this machine, and an overnight calculation ends when nobody is. Chat alerts... takes a webhook URL and posts the same sentence to Slack, Discord, Teams, or anything else that accepts a JSON POST — which is what puts it on a phone.

Nothing is installed for it (it uses urllib), and nothing leaves the machine until you paste a URL in. The payload key is chosen from the URL, because the services disagree and are unforgiving: Discord reads content and rejects a body carrying fields it does not know, while Slack and Teams read text. Only http and https are accepted, so a pasted file path cannot make the plugin read a local file.

Chat alerts have their own checkbox beside the desktop one, so you can pause room messages for a week without deleting a URL you would then have to dig out of your workspace again. Two deliberate asymmetries:

  • The tick cannot be set before a room is configured — a tick with nothing behind it would claim messages are going out when none are, and only a job ending would disprove it.
  • Saving a URL does not switch it on. Sending a job's name and host off this machine is a decision to take deliberately. Clearing the URL does switch it off, there being nothing left to post to.

When a job ends, outputs are fetched automatically — next to the input file by default — and handed to whichever plugin already claims that file type: a finished ORCA run opens straight in the ORCA Result Analyzer with no extra configuration. Opening a result clears the editor first, so the molecule you submitted does not stay on the 2D canvas beside a 3D view of what came back; you are asked about unsaved work exactly as File > New would ask. Resubmit repeats a job with one click: same host, same inputs, same resources.

6. Submitting from an input generator

ORCA Input Generator Pro and Gaussian Input Generator Pro both show a Submit to Cluster... button beside Save once this plugin is installed — without it, the button does not appear at all. It saves the input and hands it here, prefilled, with no file picker and no retyping the job name. Work arriving that way has no host in mind — the file was saved wherever you pointed the save dialog — so the wizard opens on a host that has an equal path configured where you have one, that being the machine whose results need no downloading. Pick a host yourself and it stays picked. Any other plugin can do the same through job_manager.submit_file(paths, name=""), a public API found via the host's plugin list.

7. Standalone mode

Job Manager and its Host Monitor both run without MoleditPy at all:

python -m job_manager                    # Job Monitor
python -m job_manager --host-monitor     # Host Monitor

A .bat wrapping pythonw -m job_manager (so no console window stays open) makes either one a double-click desktop shortcut on Windows.

8. Where your data lives

~/.moleditpy/job_manager/ — deliberately outside the plugin folder, since the Plugin Installer replaces that folder wholesale on update and jobs can run for days.

File Holds
settings.json host profiles, submit presets, preferences, saved command templates — no password is ever written here
jobs.pmejbs the tracked job list, plain JSON
archived/jobs_<date>.pmejbs lists written by Clear List — clearing never deletes
downloads/ fetched results, one directory per job

A folder of results nobody tracked — fetched by hand, copied off a cluster, or run before this plugin existed — can be rebuilt into a read-only job record with Rebuild list from folder: one entry per directory that holds outputs, saved beside them. Nothing in a rebuilt list can be submitted, cancelled or polled, since there is no host behind any of it.

9. What it does not do

No workflow graph. Chaining is a straight line — each job waits for exactly one predecessor. There is no fan-out, no "run C after both A and B", and no retry on failure; anything branching goes into Extra directives by hand, which Job Manager passes through and then tracks normally.

The helper queue is not a batch system. FIFO with two resource budgets and one-predecessor dependencies — no priorities, no backfill, no fair share, no reservations, no accounting.

No file browser and no allocation/accounting queries — fetch patterns decide what comes back; there is no sacct, no sinfo, no quota lookup.

10. See also

This page documents Job Manager v1.1.3, and was last updated on 2026-08-26. See REGISTRY/plugins.json for the current version.

Clone this wiki locally