v0.9.0

Changed

• TRIM.run() and SR.run() now run in an isolated directory by default. Every run works in its own private copy of the SRIM install (hard links for the read-only files, so a copy adds well under 15 % of SRIM's size; real copies of what SRIM writes) under <data dir>/runs (or work_dir=); the installed SRIM directory is never written to, so any number of runs, scripts, scans or users can run at the same time. On Linux each run also gets its own clone of the wine prefix, so cleaning up after a failure only stops that run's wine processes. The returned Results points at the run directory (results.directory); pass output_dir= to get the output files and run.yaml in a folder of your choice (the run directory is then removed unless keep=True); run_dir= reuses a srim.rundir.RunDir. A failed run keeps its directory for debugging and names it in the error.
  Migration: code that read outputs from the SRIM directory after run() (TRIM.copy_output_files(srim_dir(), dest), Results(srim_dir())) must use the returned results instead: TRIM.copy_output_files(results.directory, dest) or run(output_dir=dest). Passing srim_directory= explicitly keeps the old in-place behaviour, now under an exclusive file lock (a second run waits and prints a message after 5 s; lock_timeout= limits the wait).

• TRIM.copy_output_files copies the files that SRIM writes to its SRIM Outputs subfolder instead of moving them, so a Results of the source (the one run() returned) keeps working afterwards.

• pysrim2 run uses an isolated run directory too, so several can run at once.

• Runs no longer change the process working directory (the input writers take a directory= argument and SRIM is started with cwd=), so run() is safe in threads. Under xvfb-run every run claims its own display number (-n, via lock files <tmp>/.pysrim2-xvfb-<N>.lock) instead of racing with -a; an X server that fails to start is retried up to three times.

• Run directories without output_dir= are deleted automatically once unused for 7 days (PYSRIM2_RUN_RETENTION_DAYS, 0 keeps them; failed runs too); Results.close() / with trim.run() as results: delete one at once; pysrim2 clean runs [--older-than DAYS] [--dry-run] and pysrim2 doctor (size of the run directories) manage them. Open Results keep their run directory alive (shared lock) and pruning skips folders younger than a minute; leftovers that cannot be deleted are reported with a warning, and files shared with the SRIM install (hard links) never have their mode changed.

• Reading outputs from the installed SRIM directory (Results(srim_dir()), TRIM.copy_output_files(srim_dir(), ...)) after isolated runs gives a one-time UserWarning explaining the 0.9.0 change.

• Nothing is ever created in the installed SRIM directory: the lock for in-place runs lives in <data dir>/locks/, and copying the install waits for an in-place run that is writing; an in-place run in a read-only SRIM directory fails with a clear message. Relative work_dir= is made absolute.

Added

Documentation

• New pages docs/scans.md (scan concepts, extractors and their columns and units, precision, the result table, cache, backends, live views, exit codes, tmux over SSH) and docs/concurrency.md (isolated run directories, locks, migration from 0.8, one cache for several scans). docs/cli.md lists the new commands.
• A scan notebook (examples/notebooks/Scan.ipynb), an example scan (examples/scripts/angle_energy_scan.py and scan.yaml, run by the tests with real SRIM) and a scans section in the agent skill (skills/pysrim2/SKILL.md, section 7).

Scan definition and extractors

• srim.scan.Scan(base, grid=, points=, extract=, ions=, seed=, batch_ions=) defines a parameter scan: a cartesian grid and/or zipped points over dotted keys (ion.energy_keV, target[0].thickness_nm), points() with a stable id per point (spec_hash), and a scan: section in run.yaml read by Scan.from_yaml / written by Scan.to_yaml. srim.scan.Precision describes a precision goal for ions=. Errors name the dotted key; number_ions and seed are not scan parameters.
• srim.scan.extract: extractors turn a run's outputs into scan columns (register, get, available, resolve, combine) with predefined implant_summary, vacancy_at(depth_nm), peak_vacancy_depth, sputter_yield, backscattered_fraction, transmitted_fraction, ions_in_layer(layer_index) and energy_to_recoils.
• srim.scan precision targets: Scan(..., ions=Precision(target=, rel=|abs=, min_ions=, max_ions=, batch_ions=)) runs each point in batches of ions (batch seeds derived from the point seed) until the target's standard error (<value>_err, the batch-means estimator) reaches the goal; the value is the extractors applied to all batches merged. A point that reaches max_ions first has status max_ions and a warning; a stopped point keeps the estimate of its finished batches. batch_finished events report each decision.
• srim.scan.ScanResult: one row per scan point (parameters, id, values with <value>_err, n_ions, seeds, status, error, path relative to base_dir=, so exports hold no host paths) with to_parquet, to_csv (a folder with scan.csv and metadata.yaml), to_hdf5 and plot(x, y, hue) (matplotlib, error bars from <value>_err).
• New optional extra pysrim2[plot] (matplotlib) for ScanResult.plot; pysrim2[all] includes it.

Running a scan

• A scan folder holds a .pysrim2-scan marker (written by run_scan and prepare_slurm) and a running scan holds .scan.lock: a second run_scan in the same folder raises srim.scan.ScanBusy before it writes anything.
• srim.scan.run_scan(scan, workers=, scan_dir=, on_event=, stop_event=, retries=) runs the points of a scan in worker processes (workers defaults to the CPU count minus one; each worker reuses one private run directory). It writes a scan folder (scan.yaml, points/<id>/ with the outputs and run.yaml, result/) and returns the ScanResult. A failed point is retried once with the same seed, then reported with its error (points/<id>/error.log) while the scan goes on. Progress and results arrive as JSON-safe events (srim.scan.events: scan_started, point_started, point_finished, point_failed, worker_stats, ...) from one thread, in order. Ctrl-C (or stop_event) stops scheduling, kills the running SRIM runs (also children that ignore SIGTERM) and keeps the finished points; the others get status stopped and a point_stopped event. Workers stop with their parent (POSIX: a killed scan leaves no SRIM running; its run directories are pruned by later scans). Workers that keep dying stop the scan with an error. Python extractors may be passed as callables: scan.yaml stores them as python:<module>:<name> references, which read back for module-level functions (lambdas raise a clear error when the file is read). backend="container" runs each unit in its own container (the next item).
• srim.scan.Scan(..., batch_ions=N) (YAML scan.batch_ions) splits a point with a fixed ion count into batches of N ions, each an independent run with its own derived seed, spread over the workers; the point's values and <value>_err are the batch-means estimate, as for a precision target, and each batch is cached. The default (None) keeps one run per point. Not allowed with a precision target.
• srim.scan.run_scan(..., pause_event=): while the event is set no new unit starts (running units finish, a stop still works); each change is reported as a scan_paused event (ScanPaused, paused).
• srim.scan.run_batches(spec, n_ions, batch_ions, seed) runs a calculation as batches with deterministic recorded seeds (derive_seed) and returns MergedResults (per-ion tables concatenated with global ion numbers and a batch column, depth profiles merged as ion-count-weighted means, collisions streamed), usable like Results by the extractors and exports.

Result cache

• srim.scan result cache: run_scan(..., cache=) keeps every finished SRIM run (its output folder with run.yaml) in <scan folder>/cache (or $PYSRIM2_CACHE, or the folder given; cache=False turns it off), keyed by srim.scan.cache_key(spec, seed): a SHA-256 of the run spec that writes TRIM.IN (number_ions included) and the seed, never of provenance. A rerun, or another scan that needs the same runs, reuses them without computing them again (point_cached events; status cached); a precision rerun with a tighter goal runs only its new batches; two scans that need the same run compute it once (per-key locks, so the second waits and reuses it). Entries are written atomically, failed runs are never stored, and an entry made with another SRIM build is computed again and replaced by the new run. A point all of whose runs come from the cache has status cached (a precision point that stops at max_ions keeps that status). srim.scan.clear_cache(path) empties a pysrim2 cache folder: it removes only the cache's own entries and keeps its lock files, and it warns about anything else it finds. pysrim2 scan clean is the command-line form.

Backends

• srim.scan.run_scan(..., backend="container", image=, runtime=) runs every unit in its own container of the pysrim2 image (Podman or Docker, found like pysrim2 install; PYSRIM2_IMAGE or the release image). Events, results and cache are the same as for the local back end. The worker that runs a unit removes its container when the scan is stopped, and a Docker run gets the seccomp profile that wine needs. Without a container runtime the scan fails before any worker starts. A custom runner= is not allowed with this back end. Containers run with --platform linux/amd64 (the image is amd64-only) and --network=none.
• Job folders: srim.scan.backends.run_job(dir) and pysrim2 run-job DIR run the units of a folder (job.yaml, outputs in out/<unit>/, .done written last, error.log for a failed unit); a second run only runs what is left, so an interrupted job resumes. Exit codes: 0 all done, 1 some failed, 2 not a job folder.
• Slurm: srim.scan.prepare_slurm(scan, scan_dir, jobs=None) writes the scan folder for a cluster: scan.yaml with the base seed, one job folder per point (or jobs=N round-robin groups) and submit.sbatch, a job array that runs pysrim2 run-job per folder. srim.scan.collect_slurm(scan_dir, cache=None) builds the same ScanResult as a local run once the tasks are done (failed points carry their error.log summary; points not run yet get status missing with the error "not run yet"), and stores the finished runs in the cache. Fixed ion counts only (precision targets need a local or container run).

Command line

• pysrim2 scan run SCAN.yaml [--workers N] [--backend local|container] [--image REF] [--ui plot|dashboard|web|plain|json] [--output DIR] [--cache DIR | --no-cache] [--retries N] [--set KEY=VALUE ...] runs a scan and prints its result path. --ui plain prints a status line when the state changes; --ui json writes every event as one JSON line on stdout. When stdout is closed early (e.g. piped into head), both stop writing quietly and the scan finishes. Exit codes: 0 every point done or cached (a point that reached max_ions gives a warning), 1 some point failed or is missing, 2 bad input or nothing could start, 130 stopped by Ctrl-C (the partial result is kept). plot (the default) and dashboard are the live terminal views (see the [tui] extra below); web is the same live view in a browser (see below).
• pysrim2 scan prepare SCAN.yaml -o DIR [--jobs N] [--set ...] writes the job folders and submit.sbatch for a cluster; pysrim2 scan collect DIR [--cache DIR | --no-cache] builds the result of the finished tasks and prints a summary per status.
• pysrim2 scan clean DIR [--dry-run] removes the cache entries of a scan folder that no scan is using; --all removes the whole folder, and only one with the .pysrim2-scan marker: it refuses a symbolic link, the working directory or a folder above it, the home folder, and a folder that a running scan holds (a removal that fails exits 1). pysrim2 scan clean --cache PATH clears a shared cache.
• pysrim2 scan run reuses a scan folder whose scan.yaml describes another scan with a warning on stderr, and a rerun without seed in the same folder keeps the folder's recorded seed. A scan that fails after it started exits 1 (scan failed); only an error before the start exits 2. pysrim2 scan run and prepare take the path after --set in either order. PYSRIM2_SCAN_RUNNER prints a notice on stderr when it replaces SRIM.

Live views

• srim.ui.PlainProgress (a status line when a scan's state changes, at most every 2 s, no escape codes) and srim.ui.JsonLines (every event as one line of JSON) are event consumers for run_scan(on_event=...); pysrim2 scan run --ui plain|json uses them.
• The plain view's final line counts the points that came from the result cache as done and says how many: finished 12/12 done (12 cached), 0 failed, 0 stopped ions 2400 … (a point of which a worker ran a unit is done, as in the result table).
• Live terminal views of a scan (optional extra pysrim2[tui]: textual, textual-plotext and textual-serve; pip install pysrim2[tui]). pysrim2 scan run opens the live plot in a terminal by default: the chosen result fills in as points finish, its error bars shrink as a precision point accumulates ions, and a status strip shows done, running, queued, failed, ions, elapsed and ETA. --ui dashboard shows the overall counts and gauge, the workers, a throughput sparkline, a sortable scan-points table (click a header) and the latest warnings. --plot X,Y[,HUE] picks the axes and series (defaults: the first and second scanned parameters, the first extracted value; X,Y, for no series). Keys: p pause or resume (no new point starts; running ones finish), q (or Ctrl-C) stop cleanly (pressed again, Ctrl-C quits at once, and the scan stops in the background for at most the grace period; SIGTERM stops the scan too), v switch plot and dashboard, l the error.log of the selected or last failed point. The views redraw incrementally, work over SSH and in tmux, and never hold up the workers. Without a terminal, or without the extra, plain runs with a notice, as before. Migration: scripts that run pysrim2 scan run in a terminal and expect plain text should pass --ui plain (or --ui json for events).
• The terminal views stay open after the scan ends. The status strip says finished — press q to quit (the dashboard says it in its overall panel); every key still works, p does nothing, and q or Ctrl-C quits with the scan's exit code. pysrim2 scan run --no-wait closes the view as soon as the scan has finished, as before; ScanApp(wait=False) is the Python-side switch. A scan stopped by the user, or one that fails, still closes the view at once.
• srim.ui.tui: ScanState (the counts, workers, points, throughput and ETA of a scan, from its events) and plot (the series and error bars of a result) are pure Python, usable without Textual; srim.ui.tui.app.ScanApp is the Textual app.
• Live view in a browser: pysrim2 scan run --ui web runs the scan as --ui plain does and serves the same app as the terminal views at http://127.0.0.1:8000 (--host, --port; the URL is printed; needs the [tui] extra, which now includes textual-serve). Several tabs can watch one scan, and a tab opened late shows what has happened so far. p and q in the browser pause, resume and stop the scan (after the end they do nothing); every event is also written to <scan folder>/events.jsonl, and the requests to control.jsonl. After the scan ends the server stays up, with the final state on screen, until Ctrl-C (the notice scan finished — the view stays at <URL>; press Ctrl-C to stop it says so); pysrim2 scan run --no-wait stops it a few seconds after the end instead. Ctrl-C after a finished scan keeps the scan's exit code (not 130). Without textual-serve, or when the port is in use, plain runs with a notice. A --host other than the loopback address warns that anyone who can reach the port can pause and stop the scan: the web view has no authentication. The view also has no Origin check, so a web page open in a browser on the same machine can pause and stop the scan through the local port: use --ui web on trusted machines only.
• The browser view is robust: a server that exits at start (port in use) is a notice after one second and the scan goes on; a failing events file or control file is one notice (the view is turned off, or polling goes on).
• A SIGTERM (a plain kill) to pysrim2 scan run --ui web stops the scan like q and then ends the server and its viewers.
• Each run of the browser view starts its events file with a run line carrying a run id; a viewer that sees a new run (or a truncated file) starts its state again, so a tab left open never mixes two runs.
• The browser view's served command is quoted for cmd.exe on Windows and for sh elsewhere.
• The browser view stops completely at Ctrl-C (or after the grace period with --no-wait, or when the scan was stopped by the user): the server runs in its own process group, so an open tab's viewer is ended with it (a tab that stays open no longer leaves a server and a viewer behind), and the server and each viewer exit when the process that started them is gone.
• srim.ui.LiveView(plot=None, every=1.0): run_scan(scan, on_event=LiveView()) shows the status line and the plot of a scan in a Jupyter cell (the plot of ScanResult.plot: same axes and error bars; plot="X,Y[,HUE]"), updated at most every second and once more when the scan finishes. Each redraw runs in the scan loop: raise every for a big scan. It needs IPython and matplotlib (pip install "pysrim2[plot]" ipython; IPython is there in Jupyter already), a Jupyter session (it refuses to start in a plain script) and no widgets; a failing view warns and stops, and the scan goes on.

Events and other

• srim.scan.events.event_from_dict(data) reads an event back from its to_dict() or one line of to_json() (fields it does not know are ignored); the browser and notebook views build their state from the events this way.
• srim.rundir.RunDir (private working copy of SRIM, reset(), disk usage) and Results.directory.

Fixed

• pysrim2 run --set KEY=VALUE run.yaml works: the path after --set was read as one of its values, so the command failed with a missing run.yaml. The path also comes in either order, including a path with = in its folder name (runs/x=1/run.yaml).

• implant_summary on an empty frame returns NaN values without emitting RuntimeWarnings.

• pysrim2 run no longer leaves an empty output folder behind when the run fails.

• The agent skill names the event-file DataFrames correctly (results.exyz.df, not results.exyz).

• srim.plot.plot_vacancies no longer adds the primary knock-on atoms to the vacancy total: knock-ons are not vacancies, and the plotted curve is the sum of the element columns (matching "Total Target Vacancies" in VACANCY.txt).

• num_ions of the output tables is SRIM's ion count rounded to the nearest integer. It was truncated, so a run of 60 ions ("Total Ions calculated = 59.93") read as 59. The raw value is kept as metadata["ions_calculated"]. Values normalised by it (e.g. plot_phonons) change slightly.