feat: shell completions, with the ids and names of the jobs

[stephenc]

qex offered no completions. qex completions <shell> now writes them for bash,
zsh, fish, elvish and powershell.

The valuable part is the jobs

A job id is a uuid, and nobody types a uuid. After qex status, qex wait,
qex logs, qex rerun, qex clean, qex kill and qex cancel, the shell now offers every
job by its id AND by its name.

Each command offers the set that it accepts, and no other:

qex kill <TAB>     the jobs that operate
qex cancel <TAB>   the jobs that wait in the queue

A candidate that the command would refuse teaches the wrong command.

This matters to a person who drives an agent from a terminal. The agent writes
the full command and needs no completion, but the person who watches the work,
stops a job or reads a log types the ids by hand. Those ids are uuids that the
agent chose.

The ids cannot come from the definition of the command line

They change with each job, so the shell asks qex at the moment of the TAB, with
a hidden command: qex __complete.

That command never starts a coordinator. It reads the records on the disk.
qex list would start one, and a press of TAB is not a request to start a
process: a user who pressed TAB in a directory with no work would leave a
coordinator behind. A test holds that rule.

qex SHOWS a safe form of each name

This is a change of behaviour, and a reader with old records sees it. A job
that you named deploy prod is shown as deploy_prod.

qex now shows a name that holds the letters A to Z and a to z, the
numbers 0 to 9, and the three characters -, _ and .. Every other
character becomes _, a run of them becomes ONE _, a first character of -
becomes _, and the result stops at 128 characters.

deploy prod$(id)   ->  deploy_prod_id_
-version           ->  _version

This is at EVERY output: qex list, qex status, qex top, qex du, qex gc, the sentence that says why a job waits, the sentence that says which job
failed, the sentence that says that a record is gone, the log of the
coordinator, the completions, and the JSON of each of them. The JSON holds the
safe form as well, because a machine that reads it renders what qex gives it and
it knows no more than qex does.

The record on the disk keeps the name that you gave. qex changes no record,
and qex gc is not the thing that applies the rule: the safe form comes from
the name in the record, so the rule reaches every record at once.

A safe name goes back into a command as it stands. resolve_id finds a job
by its safe name and by its stored name, so you copy a name out of qex list
and paste it into qex status. Two names that give one safe form make that word
name more than one job, and qex gives the error that it already gives for such a
word: it lists the jobs and it asks for an id.

Why. A name is text that another agent chose. A name that holds an ESC byte,
written to a terminal by qex list, moves the cursor and writes over the text
around it. No shell and no TAB are needed for that, so the answer belongs at
each output and not at one.

A job name must not run when somebody presses TAB

An agent chooses the names of the jobs, and a person presses the TAB, so a name
is text that an attacker can write.

Each shell puts the word on the line as ONE word. A candidate that holds
build; rm -rf ~ thus becomes one argument of qex, and never a command. bash
uses printf %q; zsh and fish make the word safe themselves.

The safe form above is not a reason to stop doing that. The answer of qex __complete is text that came off a disk, and a disk is not a guarantee: a
record that another program wrote, a record of a qex that is older or newer, and
a fault in the sanitiser all reach the shell in the same way. A test puts a
stand-in qex in front of the real one and answers with 19 hostile names, and
it requires that each arrives as ONE argument that is the name itself.

The candidates also go into the reply one line at a time. compgen -W expands
its word list AGAIN, and a job named $(...) ran when somebody pressed TAB.

compopt -o filenames asks bash to do the quoting instead, and this work used it
first. bash then treats each name as a FILE NAME: a job with the name of a
directory got a / after it, a name of $HOME became a directory as well, and a
name that starts with ~ was expanded to a home directory. Each of the three
gave qex a name that no job has. printf %q has none of that behaviour, and it
was measured on bash 5.1.16 and on bash 3.2.57.

One behaviour remains, in fish only. fish reads a leading ~ as your home
directory when you press ENTER. docs/reference.md says so, and it gives the
remedy.

The hidden commands stay hidden

#[command(hide = true)] keeps a command out of the help text, and it does NOT
keep it out of the completions. A person who pressed TAB was offered qex daemon — a command that starts a coordinator in the foreground — beside qex submit. qex daemon, qex supervise and qex __complete now go out of the
completions of every shell.

What each shell gets

bash, zsh and fish get the commands, the options, and the jobs.

elvish and powershell get the commands and the options only. qex does not
test them, and a completion that nobody tested teaches a value that may not
exist.

Tests

Three tests.

1. The completions name the real commands, they hold no hidden command, and
   each shell asks qex for the correct set. The zsh test reads the job argument
   of each command, because an earlier form of this work put the zsh part after
   the line that gives the completion to the shell: the words were in the file
   and zsh never ran them.

2. The candidates come from the disk and start no coordinator, and a name that
   holds a line break is not offered.

3. A REAL bash sources the completions and calls the completion function.
   A fourth test sources the file and reads what complete -p qex holds, which
   is the one thing the others cannot see. It gives the correct
   jobs for qex status, qex kill, qex cancel and qex clean; it gives no
   job for the value of --signal and it still gives one after the flag
   --json; it does not offer a job named --version where an option goes;
   and it keeps a name that holds a space in one piece. It also gives a job the
   name bait$(touch FILE), reads the completed word back the way the command
   line reads it, and requires one argument and no file.

4. A stand-in qex answers with 19 hostile names, and bash must give each
   as ONE argument that is the name itself, and run nothing.

5. Every output shows the safe name, the record on the disk keeps the stored
   name, the round trip through qex list works, and no output writes the bytes
   of a name that holds an ESC byte.

Also here

CONTRIBUTING.md now says that a change goes on a branch and that a pull
request merges it, with the commands to do so.

-o nosort came with bash 4.4, and complete refuses the whole command when it
meets an option name that it does not know. The registration now tests the
version of bash, in the way that clap tests it for its own line. Without that,
sourcing the file on the bash 3.2 of macOS bound NOTHING and wrote an error at
each shell start.

Not changed: qex pipeline echoes the names of the stages of the file that
you gave it, and those names keep their form. A stage name is your own input in
the same command, and it is a key of the mapping that --json and the id file
give to a machine. The job that a stage becomes is a job like any other, and
every command shows its safe name.

[stephenc]

Before the merge

• Rebase on main (7927324, after docs: say what qex run is for, and what it costs #5, fix: accept a number where the configuration takes a number or text #6, fix: the coordinator reads its configuration again when it changes #7 and fix!: give the exit code that says why the job of qex run stopped #40)
• Resolve the collision in CONTRIBUTING.md in favour of main
• cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings
• 193 unit tests and 89 end-to-end tests pass on Linux, and CI is green on
  Ubuntu 22.04, Ubuntu 24.04 and macOS 14
• Drive the completions in a real bash, a real zsh and a real fish, with
  hostile job names
• An independent review of the rebased branch
• The six findings of that review, corrected and measured
• A second independent review, and its four findings
• qex shows a safe form of each name, at every output
• CI green after the corrections

What the verification found

The branch said that the zsh and fish parts were written and not run. zsh 5.8.1
and fish 3.3.1 went on to the machine and ran them, and zsh gave no job at
all. Two faults, and both are now corrected:

1. The zsh part was after the lines that give the completion to the shell, so
   nothing ever called it.
2. clap gives each job argument the action _default, which offers every file
   and every command on the machine. qex kill <TAB> gave 2674 candidates.

_qex_jobs now sits at the top of the zsh file, and each job argument names it.
zsh completes a job on the FIRST press of TAB, from an autoloaded file and from
a sourced one.

Three more:

• bash wrote the name to the line as it stood. A job named
  x; touch FILE put ; touch FILE on the line beside qex, and the next
  press of ENTER ran it. bash now makes the word safe with printf %q.
• A name that holds a control character became two candidates, or it moved
  the cursor of the terminal. qex __complete no longer offers such a name.
• elvish and powershell still offered qex daemon. They no longer do.

What was driven, and how

Each shell loaded the completions and a real TAB was pressed: bash 5.1 and zsh
5.8.1 in a pty, and fish 3.3.1 through complete -C. The three of them complete
a subcommand, a job id, a job name, and they give nothing when there is no job.
qex info --no-start said no coordinator after every one of those presses.

The hostile names. 22 job names went into a scratch queue: $(touch FILE),
`touch FILE`, x; touch FILE, x | touch FILE, x & touch FILE,
>redirect FILE, ${IFS}x, $HOME, ~/tilde, glob-*, ?, [abc],
\backslash\, !hist, quote"double, quote'single, has space inside, a
name with a line break, a name with a tab, -leading-dash and a name of 2000
characters. No press of TAB made a file in any shell. A TAB and then an ENTER
runs qex status with the name as ONE argument, and still makes no file.

What is not covered

• elvish and powershell have no shell on this machine. They get the commands
  and the options only, and no test drives them.
• The order of the candidates, newest job first, has no test.
• qex submit --needs <id> and qex submit --after <id> take a job and offer
  none. They are option values.
• qex clean completed and qex clean done take a state word, and the
  completion offers the jobs and not those two words.
• fish reads a leading ~ as your home directory when you press ENTER, so
  a job name that starts with ~ arrives as a path. docs/reference.md says
  so and gives the remedy. bash and zsh do not have this behaviour.
• qex status -leading-dash<TAB> offers nothing, because a word that starts
  with - is an option and not a job.
• The measurement above of printf %q against compopt -o filenames is in the
  comment below.

[Copilot]

Pull request overview

This PR introduces first-class shell completion support for qex, including dynamic TAB completions for job ids and job names by querying qex __complete at completion time (without starting a coordinator). It also updates documentation and adds end-to-end coverage for the new completion behavior and safety constraints.

Changes:

• Add qex completions <shell> (generates scripts via clap_complete) and hidden qex __complete <what> (prints job candidates from on-disk records).
• Inject dynamic job completion logic for bash/zsh/fish, including filtering hidden subcommands out of generated completion output.
• Add e2e tests that validate shell script contents and that “real bash” completion does not execute malicious job names; update docs/CONTRIBUTING accordingly.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/e2e.rs	Adds e2e tests for completion correctness, safety, and coordinator non-start behavior.
src/main.rs	Implements completions generation and dynamic completion injection/removal of hidden commands.
src/commands.rs	Adds hidden __complete command to print completion candidates from disk.
src/cli.rs	Adds CLI plumbing for completions and __complete.
README.md	Documents how to install shell completions.
docs/reference.md	Adds completions docs and command synopsis entry.
CONTRIBUTING.md	Adds guidance to work on branches + open PRs.
Cargo.toml	Adds clap_complete dependency.
Cargo.lock	Locks clap_complete and updates dependency graph.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[stephenc]

The six findings

Finding 3 first, and the review is right. My reason for rejecting
printf %q was an artefact of MY MEASUREMENT, not a behaviour of bash. I read
the raw byte stream of a pty with the cursor movements stripped, so a word that
readline REPLACED looked like a word that readline had appended. qex status ~/til<TAB> shows as ~/til\~/tilde… in that stream and the shell holds
~/tilde…. I proved it by pressing ENTER and reading what qex received, which
is the measurement I should have made the first time.

Measured on this branch, in a real bash, with TAB and then ENTER, and the name
that qex received:

typed	compopt -o filenames	printf %q
zzd, with a directory zzdocs	zzdocs/ — refused	the job
$H, name $HOME	$HOME/ — refused	the job
~/til, name ~/tilde$(…)	/home/…/tilde$(…) — refused	the job
semi, has, al, glob, [ab, !hi, \ba, ?, colon	the job	the job

printf %q is correct in every case measured, and compopt -o filenames is
wrong in three. bash now uses printf %q alone. That removes findings 1 and
2 for bash, and it removes the bash 4.0 floor as well.

Finding 1 (a directory of the same name) and finding 2 (a leading ~)
are gone for bash. fish still expands a leading ~ on ENTER, and
docs/reference.md now says so and gives the remedy.

Finding 4. complete -F _qex_with_jobs … now has three assertions: the
registration names _qex_with_jobs, that function runs _qex and then
_qex_jobs, and the whole set of job assertions. Binding _qex instead now
fails the test.

Finding 5. The guard read every word that starts with a dash. It now names
the options that TAKE A VALUE, and that list comes from the command line
definition itself, so an option that a later change adds needs no second edit:

case " --context --cwd --grace --grep --head --id-file --lines --max-matches \
      --older-than --signal --state --tail --timeout --under -C " in ...

Measured: qex status --json <TAB> and qex status --show-env <TAB> give the
jobs; qex kill --signal <TAB> and qex kill --grace <TAB> give no job.

Finding 6. qex clean is in the set that offers every job, in all three
shells. Measured: bash, zsh and fish each complete qex clean zzd<TAB> to
zzdocs and qex clean al<TAB> to alpha. qex clean completed and
qex clean done take a state word, and the completion does not offer those two
words; that is a gap and not noise.

The sentence about the line break. Corrected. The code drops every control
character, so the text now says that, and it gives the reason for the escape
byte as well as for the line break.

Mutation testing, run again

22 mutations, 21 caught. Two that the review added are now caught:

• bind _qex in place of _qex_with_jobs — caught (finding 4).
• case "$cur" in -*) return 0 removed — caught. A job now carries the name
  --version, and the test requires that qex status --<TAB> does not offer
  it and that qex status <TAB> does.

Also caught: going back to compopt -o filenames; making the guard read every
dash word; taking clean out of the set; and putting the flags into the list of
options that take a value.

One declared survivor stays: out.reverse(), the newest job first. It has
no test.

The numbers

193 unit tests, 89 end-to-end tests, cargo fmt --all -- --check clean,
cargo clippy --all-targets -- -D warnings clean.

[stephenc]

The sanitised name, and the second review

The rule, and one measurement that changed it

job::safe_name is the one function. Each character outside
A-Z a-z 0-9 - _ . becomes _, a run of them becomes ONE _, a first
character of - becomes _, and the result stops at 128 characters.

The instruction said "replace each character outside the set with _" and gave
deploy prod$(id) -> deploy_prod_id_. Those two do not agree: one character
for one _ gives deploy_prod__id_, with two underscores. I implemented the
COLLAPSING form, because it is the one that the example and the test name.

A second correction to the instruction, also by measurement: a b and a-b
do not give one safe form. - is inside the set, so a-b maps to itself. The
collision test uses x y and x_y, which do collide.

Where it is applied

One function at the output boundaries. commands::for_display converts a record
once, at the point where it stops being a record and becomes something a reader
sees; qex list, qex status and qex wait each call it once, so the table,
the sentences and the JSON all come from that one value and no name is
sanitised twice. JobStatus::display_name covers qex top, qex du, qex gc
and the ambiguous-name error. The sentences that BAKE a name into another
sentence are done at the producer, because a sentence cannot be taken apart
later: sched.rs (the lock reason, the dependency reason, the broken-dependency
reason, and the two log lines of the coordinator), lifecycle.rs and
history::describe_missing.

Measured on a real queue: the record on the disk holds deploy prod$(id), and
qex list, qex status, qex top, qex du --json, qex gc --json,
qex wait --json and the completions all hold deploy_prod_id_.

The collision falls through cleanly, with no new error:

qex: `x_y` names 2 jobs. Give the id of the job that you want, or delete the
old jobs with `qex clean done` and start again.
  1f09fecd-… x_y
  37ca4385-… x_y

Not changed, and I want this seen rather than assumed: qex pipeline echoes
the names of the stages of the file that you gave it, and those keep their form.
A stage name is your own input in the same command, and it is a KEY of the
mapping that --json and the id file give to a machine; changing it would break
a consumer. The job that a stage becomes is a job like any other, and every
command shows its safe name.

The second review

Finding 1 is correct and it is now fixed. -o nosort came with bash 4.4,
and complete refuses the whole command when it meets an option name that it
does not know. The registration now tests BASH_VERSINFO in the way that clap
tests it for its own line, 90 lines earlier, and it falls back to
complete -F _qex_with_jobs -o bashdefault -o default qex. -o nosort stays
for a bash that has it, because it is what keeps the newest job first.

The false sentence is gone from the code comment and from the pull request body.
It now says what was measured: bash 5.1.16 and bash 3.2.57.

The root cause is closed. A new test sources the file in bash, runs
complete -p qex, and requires that the shell holds -F _qex_with_jobs, with
an empty stderr and set -e. Dropping the version gate now fails a test;
binding _qex now fails a test.

Finding 2. Both stale comments are gone.

Finding 3. The ESC case is asserted on the BYTES, in
every_output_shows_the_safe_name, across qex list, qex status, qex wait,
qex du, qex gc, qex top --once, qex __complete, the JSON of each, and
the log file of the coordinator. The safe-set rule did supersede is_control(),
and the ESC assertion survived the replacement.

Finding 4. Both body edits are made.

The note about $ and ~ in zsh and fish is in docs/reference.md.

The quoting is still under test

Sanitising means the real qex __complete no longer answers with a ;. That is
not a reason to stop making the word safe, and it is not a reason to stop
testing it. A new test puts a qex of its own in front of the real one on the
PATH and answers with 19 hostile names — $(...), backticks, ;, |, &,
quotes, a trailing lone backslash, ${IFS}, $HOME, ~/tilde, !hist, an ESC
byte, café 日本語, and a name of 2002 characters. Each must arrive as ONE
argument that is byte-identical to the name, and nothing may run. Removing
printf %q now fails that test.

Mutation testing

29 mutations, 28 caught. The survivor is the declared one: out.reverse(), the
newest job first, which has no test.

New this round and caught: the version gate; the registration binding; each part
of safe_name (the character set, the collapse of a run, the leading dash, the
128 limit); resolve_id matching the safe form; for_display for the name and
for the group name; and the stored name at qex top, qex du, qex gc, the
lock reason, the dependency reason, describe_missing and the log of the
coordinator.

The numbers

193 unit tests, 91 end-to-end tests, cargo fmt --all -- --check clean,
cargo clippy --all-targets -- -D warnings clean.

[stephenc]

A leading ~, found by macOS CI

bash_keeps_a_hostile_candidate_in_one_word failed on macOS 14 and passed on
Linux, and the difference is real: printf %q escapes a leading ~ on bash
5.1 and it does NOT on bash 3.2. A job named ~/tilde thus arrived as
/Users/runner/tilde on the bash of macOS.

The completion now makes that character safe itself:

printf -v candidate %q "$candidate"
case "$candidate" in "~"*) candidate="\\$candidate" ;; esac

Measured in both directions on bash 5.1: with %q alone the word is
\~/tilde, and the extra test does not fire; with the escape missing on a bash
that does not escape it, the word becomes \~/tilde all the same. eval gives
~/tilde in each case.

Two tests, because one of them cannot see this on Linux.
bash_keeps_a_hostile_candidate_in_one_word measures the outcome, and it can
only measure the bash of the machine that runs it — the mutation survives on
bash 5.1 and macOS CI is what caught it. the_completions_hold_the_commands_of_qex
now asserts the rule in the generated text, so the mutation fails on every
machine.

docs/reference.md now names ~ and $ together for zsh and fish, which
expand a word that the user typed before any candidate is compared. bash is
correct in both.