Skip to content

feat: SRS in LPSE (and 1d support) - #319

Closed
physicistphil wants to merge 54 commits into
ergodicio:mainfrom
physicistphil:phil/lpse-srs-dev
Closed

feat: SRS in LPSE (and 1d support)#319
physicistphil wants to merge 54 commits into
ergodicio:mainfrom
physicistphil:phil/lpse-srs-dev

Conversation

@physicistphil

Copy link
Copy Markdown
Contributor

Summary

Adds stimulated Raman scattering (backward SRS) to the envelope-2d (lpse2d) solver. Until now the solver evolved the EPW potential against a prescribed pump with a TPD source only; the E1 scattered-light field existed in the state vector but was never advanced. This PR evolves E1 with a paraxial finite-difference solver and closes the loop by adding the SRS source to the EPW equation, so backscatter grows self-consistently from noise (or from an optional injected seed).

The physics is a direct translation of the raman.solver = 'fd' branch of lpse-matlab (m201805_matlabLpse_v11.m); line references to the MATLAB source are kept in the code comments so the two can be diffed by hand.

What's added

Raman light solver (adept/_lpse2d/core/raman.py, new)

Evolves the scattered-light envelope at w1 = w0 - wp0:

dE1/dt = i c²/(2 w1) ∇⊥² E1
       + i w1/2 (1 - wp0²/w1² · n/n_env) E1
       - i e/(4 w0 me) conj(∇²φ) E0
       + seed injection (optional)

with the cross-derivative terms of the 2D paraxial operator, and the same staggered explicit update as MATLAB's lightSplitStep (real part from the RHS at t, imaginary part from the RHS at t + dt/2).

SRS source in the EPW equation (core/epw.py)

srsSource = i e wp0/(4 me w0 w1) · (n/n_env) · E0·conj(E1), added to the potential each step. E1 is high-k filtered before the product (MATLAB's isSuppressHighKSource, cutoff 1.2 × k1_max) so only wavevectors near the light-wave envelope contribute; the pump is prescribed, so it is not filtered, matching the MATLAB static-laser path.

Sub-cycling and stability (helpers.py, datamodel.py)

The light update is conditionally stable (dt < ~dx² w1/c²), so it is sub-cycled inside each EPW step with the EPW potential held fixed. The number of sub-steps is derived from the stability bound generalized to 2D, or can be pinned with grid.light_substeps — a value that violates the bound raises rather than silently going unstable. Absorbing boundaries are applied every sub-step, since light crosses the absorber at ~c.

Optional Raman seed (drivers.E1)

A two-point antisymmetric injector at x = xmax - offset launches a -x-propagating wave at the local k1, with a configurable turn-on ramp and an optional 4th-order super-Gaussian transverse profile. The default offset (1.6 × boundary_width) keeps the injector clear of the absorber's tanh skirt, and a closer one warns. If the density at the injector is above the w1 critical density the seed is evanescent, so setup fails with a message pointing at the three ways out (lower density.max, move offset, or drop E1 and run noise-seeded).

Diagnostics

With SRS on, the default time series gains e1_sq and reflectivity — the latter is sqrt(eps1)·<|E1_y|²>_y / E0_source² at a probe on the low-density side, with the sqrt(eps1) factor accounting for the reduced group velocity relative to the vacuum pump. make_series_xarrays is now generic over whatever keys the save function returns instead of hard-coding e_sq/max_phi.

Quasi-1D (ny = 1) support

1D SRS is the cheap configuration to run and the one the MATLAB srs_1D case uses, but the field-save path assumed ≥2 transverse cells: interpax.interp2d returns NaN off a single y-node and RegularGridInterpolator rejects a single-node axis (filling 0). Both silently blanked every field artifact while the scalar series stayed valid. Added an x-only interpolation path for ny == 1 in the in-solve field saver and the background-density save, and adjusted the plotting to emit line plots vs kx instead of empty kxky maps.

Also in passing: density.basis: uniform now honors a val key instead of always returning 1.0, and the complex-dtype check in make_field_xarrays tests the actual float view rather than assuming complex128.

Config surface

terms:
  epw:
    source:
      srs: true          # new; default false, existing configs unaffected

grid:
  light_substeps: 8      # new, optional; derived from the stability limit if omitted

drivers:
  E1:                    # new, optional; without it SRS grows from the EPW noise source
    intensity: 1.0e+12W/cm^2
    delta_omega: 0.0     # fraction of w1
    turn_on_time: 10fs
    offset: 5um          # defaults to 1.6 * boundary_width
    yw: 20um             # omit for uniform in y

Example config: configs/envelope-2d/srs.yaml — noise-seeded backward SRS on a 0.18–0.28 n_c linear ramp, the srs_1D case.

Validation

tests/test_lpse2d/test_srs.py:

  • test_srs_growth_rate (parametrized 2D and ny = 1) — noise-seeded homogeneous SRS. Fits the log-slope of the EPW energy over the late-time window and compares to the analytic backward-SRS rate gamma0 = k v_os/4 · wpe/sqrt(w_ek w_s), evaluated at the phase-matched k from a fixed-point solve of the Bohm–Gross/EM dispersion pair, with the pump wavenumber snapped to the FFT grid the way the solver launches it and the local density swelling folded into v_os. Agrees to 35%.
  • test_srs_seed_propagation — pump and noise off, seed only. Checks the injected wave travels in -x, that its measured wavenumber matches the local k1 to 5%, and that its amplitude matches the injector calibration E1_source · sinc(k1 dx) / eps1^(1/4) to 30%.

Notes and limitations

  • The pump is prescribed, so there is no pump depletion. E0 is reconstructed from the driver each step and is unaffected by E1, so reflectivities are only meaningful in the undepleted regime. Coupling depletion back into E0 is the natural follow-up.
  • Cost. Sub-cycling means light_substeps extra RHS evaluations per EPW step, each with several FFT-free stencil passes plus one ifft2 of the potential Laplacian per step. On the shipped example config this is single-digit sub-steps; short dx at fixed dt raises it quadratically.
  • The multi-color pump (drivers.E0.num_colors) composes with this for free — the Raman coupling consumes E0 as a field, so broadband SRS works without further changes, though it is not exercised by a test here.
  • The 2D SRS path shares the noise-seeded growth test with the 1D one; there is no dedicated test of the transverse (cross-derivative) terms yet.

Known issue, pre-existing and not addressed here: drivers.E0.shape: arbitrary (the learnable amplitude/phase driver) looks broken on main. #168 broadcast the driver output to (num_colors, ny) and switched laser.Light.laser_update to index it as [i, :], but ArbitraryDriver.__call__ overrides UniformDriver.__call__ and still returns 1-D phases/intensities (modules/driver.py:174-178), so that path would raise on the first laser update. uniform/gaussian/lorentzian all inherit the broadcasting __call__ and are fine, and no shipped config or test exercises arbitrary, which is presumably why it has gone unnoticed. Flagging it rather than fixing it here to keep this PR to SRS — happy to fold in the one-line broadcast fix if reviewers would rather have it in the same change.

Base branch

These two commits currently sit on top of the osiris-wrapper line (#279), but they touch a disjoint set of files (adept/_lpse2d, configs/envelope-2d, docs/, tests/test_lpse2d), all of which are identical to main — so this branch is cut fresh from main and does not depend on #279.

🤖 Generated with Claude Code

physicistphil and others added 30 commits May 26, 2026 13:01
functions (should probably be moved to another branch). Added density
and temperature profiles. Fixed LaTeX issues
regenerates from the saved NetCDF artifacts alone — no rerun, no raw
MS/ tree. Made list_diagnostics/load_series/load_hist_energy dispatch
between the MS/ HDF5 tree and a binary/ NetCDF dir, made field-energy
source-agnostic, and now persist HIST/energy.nc in save_run_datasets.
Added 3 tests
- regen harness: rebuild the full canned plot set from saved NetCDFs (no rerun)
- f(p) and delta-f lineouts; temperature profile from phase-space Maxwellian fits
- number-density profiles (initial/final/late-mean); 2-panel equal-aspect omega-k
- phase-space & spacetime: space on x-axis, cropped to box, log-contrast floor
- proper-LaTeX titles (prose vs math)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ion+reflection plot (should probably be moved to osiris-lpi repo)
sync-up.sh script doesn't delete ongoing osiris runs
BaseOsiris.write_units() previously returned {} so OSIRIS runs logged an empty units.yaml. Derive the physical reference scales (wp0, tp0, n0, v0, x0, c_light, beta, box_length, sim_duration) from the deck's simulation.n0 (density) or simulation.omega_p0 (frequency); when both are present, n0 wins, as in OSIRIS. This mirrors the canonical key set the other adept solvers emit so OSIRIS runs are comparable in MLflow.

Adds skin_depth_normalization and skin_depth_normalization_from_frequency to normalization.py. OSIRIS has no single global reference temperature (species carry per-species thermal momenta), so the temperature-dependent keys (T0/nuee/logLambda_ee) are omitted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Given osiris.density.gradient_scale_length, BaseOsiris scales the simulation
box so the deck's linear density ramp realizes that gradient scale length at
the reference density (default n_c/4), mirroring adept's _lpse2d / kinetic_srs
grid sizing. The transform is a single spatial scale factor applied to
space.xmin/xmax, every profile.x, and every diag_species phase-space window;
grid.nx_p scales too, holding dx fixed (rounded up to a multiple of
node_number(1)). dt/tmax are untouched, so CFL is preserved. 1D decks only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Convert each diagnostic's MS/ HDF5 dumps into binary/*.nc as the run
produces them, instead of one monolithic read-all/write-once pass at job
end. This overlaps the (hours-long) PIC compute, reads each dump while
still warm in the node page cache rather than re-opening ~700k tiny files
cold off Lustre, and bounds conversion memory to a single dump instead of
the whole stacked (t, x) series. Addresses the I/O wall-time and
conversion-memory issues in osiris-lpi/postproc-performance.md (the
pcolormesh/fft2 plot-time OOM is independent and unaffected).

New adept/osiris/stream.py:
- StreamWriter: append-only (t, ...) NetCDF writer using an unlimited t
  dim grown one slot per dump, so the file is sized to exactly the dumps
  produced (no deck-derived size guess, no trailing-fill trim) and a
  restart resumes after the on-disk slots. Schema matches
  io.series_to_dataset, so load_series_nc / plots / regen are unchanged.
- convert_diagnostic_streaming: Stage A, memory-bounded conversion at
  job end.
- StreamConverter: Stage B, a best-effort daemon thread that drains
  completed dumps live (processing dump N only once N+1 exists, to avoid
  partial reads); a final sweep picks up the last dump. RAW diagnostics
  (variable particle count) stay on the batch concat path. Fully
  failure-isolated: never aborts the OSIRIS run, and the batch path
  remains the safety net.

Wiring: run_osiris spawns/finalizes the watcher around the existing
subprocess (finalize before error handling so it can't mask a failure);
base/post thread osiris.stream_convert + stream_poll_s through;
save_run_datasets reuses the watcher's files and stream-builds any it
missed. load_series_nc now flags a dim autoscaled only when its bounds
actually move (the streamer always records them).

On by default (osiris.stream_convert defaults true).

Tests: tests/test_osiris/test_stream.py covers Stage-A equivalence vs
load_series/batch, autoscale preservation, restart-resume, the N+1
hold-back rule, the live watcher thread, an end-to-end run_osiris run,
RAW-skip, and the reuse paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OSIRIS has no asynchronous diagnostic I/O: every dump is a synchronous barrier
in the time loop, so on Lustre the ranks stall on each collective write
(~halving utilization at full dump cadence). `osiris.stage_root` works around
it as poor-man's async I/O: OSIRIS writes to a node-local /dev/shm ramdisk
(RAM-speed, no stall) and a background drainer mirrors each completed dump to
the durable run_root and reaps it from the ramdisk, so the RAM footprint stays
bounded by the poll interval rather than the whole run.

- stream.py: StreamConverter gains a `persist_dir` drain mode — mirrors grid
  AND raw dumps to persist_dir/MS, reaps the scratch, still streams grid .nc.
  persist_dir=None is byte-for-byte the old in-place behavior.
- runner.py: `stage_root` runs OSIRIS on the ramdisk, returns the durable dir
  as run_dir (post.py unchanged), syncs stragglers + reclaims the ramdisk at
  job end. Also ignore srun's spurious "couldn't chdir ... going to /tmp
  instead" launcher warning, which was false-positiving the exit-0 error guard.
- base.py: wire osiris.stage_root through.
- docs/osiris-adept-usage.md + tests.

Validated on Perlmutter (1024 ppc, nmin0.12, tmax=600): CPU 16-rank
250.7->207.0s (-17%), GPU 1xA100 128.8->123.7s (-4%); /dev/shm held bounded
while 3617 dumps drained to durable storage, full SRS post-processing intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add transverse_field_boundary_slabs(), a memory-bounded companion to
efield_lr_components for the boundary-light diagnostics (laser budget,
direction-split spectra/spectrogram) that only sample a thin slab at each
box edge. It loads each raw transverse field one at a time, slices it to the
two edge slabs, and frees the whole-grid array before the next — so the full
(t, x) left/right-going grid (tens of GiB per field on the long, wide SRS
runs) is never materialized. The Riemann split is local, so slice-then-combine
equals combine-then-slice (locked by test_boundary_slabs_match_full_split).

Also export render_cells / boxcar_downsample as public names so downstream
spectrograms can decimate to on-screen resolution before pcolormesh the way
plot_omega_k already does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
physicistphil and others added 24 commits July 1, 2026 15:29
Prepares the general OSIRIS diagnostics for 2D decks that dump only
spatially-averaged fields plus Poynting lineouts (e.g. R-Follett):

- field_energy_components resolves -savg/-tavg field variants
  (_resolve_fld_diag), integrates over all spatial dims so 2D
  (t,x2,x1) series work, and emits per-component e1/e2/e3 energies.
- plot_epw_energy: the longitudinal (e1) field energy = electron
  plasma wave energy, added to save_canned_plots as
  epw_energy_vs_time.png (works for 1D and 2D).
- _total_field_energy sums only real field components
  (_is_field_component_dir), excluding s1 Poynting lineouts / slices
  that also live under FLD/; the per-dump reader falls back to the sole
  dataset when a savg dump names it differently.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The parser only recognized double-quoted strings, tracking string state
by toggling on `"` alone. Fortran single-quoted values (the common OSIRIS
convention, e.g. ext_fld='none') fell through _parse_atom's "keep as-is"
branch and were stored with their quotes, then double-wrapped on render
("'none'") — which OSIRIS rejects. Single quotes were likewise invisible
to _strip_comment, _split_top_commas, and the value scanner, so a `!`,
`,`, or `}` inside a single-quoted string (e.g. math_func expressions)
was mis-parsed.

Make all four string-state trackers handle single and double quotes
symmetrically via a shared _QUOTE_CHARS and an active-quote-char variable
(only the matching quote closes the string). The renderer already emits
double quotes, so single-quoted input now normalizes and round-trips
cleanly. Adds tests for single-quote parsing, render normalization, and
literal `!`/`,` inside single-quoted strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OSIRIS can exit non-zero yet have written usable data — most commonly a
segfault on MPI/CUDA teardown *after* "Simulation completed" (seen on
Perlmutter 2D CUDA runs), but also a mid-run death with dumps on disk.
Previously run_osiris hard-raised on any non-zero exit (or OSIRIS-reported
error), so __call__ aborted and post_process never ran — discarding a fully
completed run's diagnostics.

Now: on a non-zero exit / OSIRIS error, check whether the run produced output
(MS/ dumps, binary/ NetCDFs, or HIST/) via _run_produced_output(); if so, log
the error as a WARNING and fall through so the caller still consolidates
binaries and generates plots from the (possibly partial) data. Only a run that
produced nothing still raises. result now carries "crashed".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Multiple line/report diagnostics of the same field share one MS/ directory and
differ only by an index in the base name (e.g. two 's1,...,line' reports ->
s1-tavg-line-x2-01 / -02). _iter_from_name reads both -x2-0N-000000.h5 as
iteration 0, so keying a series by directory stacked them into one broken
NetCDF, losing the entrance/exit distinction the laser-transmission budget needs.

io: group a dir's dumps by base (_dump_base/_series_dumps); list_diagnostics
exposes each base as its own diagnostic via a synthetic <dir>/<base> handle;
_sort_dumps resolves that handle and selects one series (bare multi-series dir
raises). load_series / _diag_is_raw / convert_diagnostic_streaming compose
through _sort_dumps, so batch consolidation writes one NetCDF per series.
stream: the concurrent converter skips multi-report dirs (can't stream several
series as one) and leaves them to the batch pass. Single-series dirs unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e h5

In ramdisk-staging mode the converter mirrored every completed grid dump
to the durable MS/ tree before deleting the scratch copy, so a staged run
still paid one inode + one copy per dump on the persist filesystem (~140k
h5 files per production SRS run). With osiris.stage_discard_h5: true the
grid dump is deleted after being appended to the streamed NetCDF and the
binary/*.nc become the only copy; RAW dumps are still mirrored (the batch
path builds their NetCDFs from the HDF5). Dumps whose stream failed are
left in place for the runner's final sync, so partial failures remain
recoverable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
list_diagnostics only walked MS/ whenever it existed, so under
stage_discard_h5 (MS/ holding only RAW) every grid diagnostic was
invisible to save_run_datasets and the plotting passes — the first
discard-mode run uploaded 2 of 14 binary artifacts and 2 of 53 plots.
Discovery now merges run_dir/binary NetCDFs for any diagnostic with no
raw dumps on disk, and the no-MS/ fallback prefers a run dir's binary/
subdir. final_iter falls back to scanning the whole MS/ tree (RAW dumps
carry iteration numbers) and field_energy_final is computed from the
last streamed FLD/*.nc slice when no FLD h5 exists. Regression tests for
the discard reap and the discovery/batch consolidation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A full phase-space history decompresses to tens of GB in memory (x1gamma_q1
is ~48 GB: 11705 x 1000 x 1024; ~0.19 GB gzipped on disk, sparse ~250x), and
eager-loading one per plot OOM'd node bundles running many sims at once. Load
only what each plot needs:

- io.load_series / load_series_nc gain t_indices= (read just those time
  slices) plus a series_len() helper; a new open_series() context manager
  yields a lazily-backed array whose isel(t=it) reads one dump on demand, for
  consumers that walk the whole time axis reducing per dump.
- save_canned_plots' phase-space plots load the final dump + evolution panels
  via t_indices; plot_phasespace_evolution samples the same set in both call
  paths and takes its colour scale from the panels only; plot_profile gains
  avg_window= so a slice-selected series keeps the intended mean window.
- _decorate copies shallow (a deep copy materialized a lazily-opened series
  just to relabel its axes).

Docs: the usage guide documents the sliced/lazy loading convention for new
plotters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMRKMZ3bRnUqLBjVtw7QJM
write_units now derives w_laser (rad/s), laser_wavelength (nm), laser_a0,
and laser_intensity — the peak intensity of a linearly polarized drive in
W/cm^2 (eps0 c E0^2/2 with E0 = a0 m_e c w_laser/e, i.e. the ICF convention
I * lam_um^2 = 1.37e18 * a0^2) — from the deck's antenna / zpulse_speckle /
zpulse section and the reference wp0. Laser-less decks are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The engine convention for _vlasov1d (and _vlasov2d, _pic1d) is the
RMS/standard-deviation thermal speed: Maxwellian exp(-v^2/2) at T=1,
L0 = lambda_De, code wavenumber k*lambda_De. normalization.py:91 was the
sole outlier with v0 = sqrt(2 T0/m_e), which made every logged physical
unit (v0, x0, c_light, box_length) wrong by sqrt(2), every dimensional
string input (um box sizes, gradient scale lengths, laser k0) off by
sqrt(2), and a 'normalizing_temperature: 2000eV' run physically a
4000 eV plasma. Dimensionless numeric-input dynamics were unaffected.

Also fixes a sign error in the NRL Coulomb logarithm
(log(n^0.5 / T^-1.25) == log(n^0.5 * T^+1.25)), which produced
logLambda_ee = -11.8 and a negative logged nuee at 2000 eV / 1.5e21cc;
the correct values are +7.22 and a positive rate.

vth_norm() is deliberately untouched: its only callers are in vfp1d,
which is self-consistently built on the sqrt(2T/m) convention.

Adds tests/test_vlasov1d/test_units_boundary.py: dimensional input in,
dimensional quantity out, checked against CODATA constants and the NRL
formulary - the units-boundary test class whose absence let this bug
survive every existing suite.

See VLASOV1D_CONVENTIONS_AUDIT.md (F1, F2) and
ADEPT_CONVENTIONS_AUDIT.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…in write_units

_tf1d re-implements the debye normalization inline; it carried the same
v0 = sqrt(2T/m) outlier (its engine, like _vlasov1d, runs in sqrt(T/m)
units) and the same log(n^0.5 / T^-1.25) sign error. Diagnostics-only:
the logged units were wrong; dynamics are unaffected (grid beta is
written but never consumed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…T0/mass

- Dougherty.compute_vbar returned the raw first moment n*u while its
  callers (find_self_consistent_beta, the drag term) treat it as a mean
  velocity. The drag therefore centered on n*u and momentum was not
  conserved wherever n(x) != 1. Now returns sum(v f)/sum(f).
- The Krook target Maxwellian was hard-coded to exp(-v^2/2) (T=1,
  m=1): any species with T0 != 1 was dragged toward T=1. It is now
  built with variance T0/mass from the species' bulk parameters, which
  are threaded through cfg.grid.species_params (new T0 entry).
- New test drives the collision operator in isolation with a 50%
  density modulation and a finite drift: per-cell density, momentum,
  and energy conservation, and the relaxed mean velocity must equal u0
  (the old operator drifts it toward n*u0).

Audit findings F4, F5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rgy monitor

- Field moments p and q were centered on the raw first moment n*u;
  they are now centered on the true mean velocity. The first moment is
  saved under the honest name 'j' and 'v' is now j/n (previously 'v'
  held the flux while the same integrand was called 'mean_j' in the
  scalars).
- The field-moment '-flogf' computed +int f log|f| dv, the exact
  negative of the scalar of the same name; both now agree.
- Adds mean_kinetic_energy / mean_field_energy / mean_total_energy
  scalars (electrostatic energy monitor, with the 1/2 factors). A
  conservation monitor of this kind would have caught the sqrt(2)
  convention bug far earlier.

Audit findings F6, F7, F12.2.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dx was xmax/nx and the interpolation period was xmax; both are now the
box length (xmax - xmin). Latent for all shipped configs (xmin: 0.0)
but wrong spacing, kx grid, and advection wrap for any xmin != 0.

Audit finding F11.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GridConfig.c_light (set only by wavepacket.yaml) was never read -
beta/c_light are always derived from the normalization. The
AmpereSolver class docstring claimed j = sum_s (q_s/m_s) int v f dv;
the code correctly has no 1/m_s.

Audit findings F12.1, F12.4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ey driver pair was a backscatter-matched triad under the old
c_hat = 11.30 (w_pump - w_seed = 1.16 = old EPW frequency) with
placeholder k0 = 1.0 on both drivers. Retuned to a matched triad under
the corrected c_hat = 15.984: pump (k0 = +0.162949, w0 = 2.79), seed
(k0 = -0.086080, w0 = 1.700942), EPW at k*lambda_De = 0.249. Matching
condition and conventions documented in the config.

Audit finding F3.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- config.md gains a 'Normalization convention' section: v0 =
  sqrt(T0/m_e) (sigma convention), L0 = lambda_De, k in 1/lambda_De,
  Maxwellian exp(-v^2/2) at T=1, Bohm-Gross w^2 = 1 + 3k^2.
- Species v0/T0 documented as code-units-only (they bypass
  normalize()); drift and thermal width now share the same unit.
- FP/Krook 'baseline' documented as a rate in units of wp0 (no 2pi),
  with the newly logged nuee_norm as the reference scale, and the O(1)
  caveat between the Dougherty nu and the NRL nu_ee.
- Super-Gaussian alpha documented (code comment + config.md): it fixes
  <v^4>/<v^2> = 3 T0/mass for all m; the variance equals T0/mass only
  at m=2 (x1.24 at m=3, x1.37 at m=4). Documentation only, per review.

Audit findings F8, F9, F10 (nuee_norm logging itself landed with the
species_params change in an earlier commit).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The fixtures locked in the sqrt(2)-wrong units (c_light 11.302, v0
2.652e7 m/s, x0 12.14 nm, negative nuee). Regenerated under the
corrected normalization and the new config surface: c_light 15.984,
v0 1.876e7 m/s, x0 8.584 nm, logLambda_ee +7.22, positive nuee, new
nuee_norm, species_params T0, c_light knob removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-mortem of job 56005549 (srs-Ln100-Te4-5x-ions): at the every-3-step
field cadence the per-run drainer thread sustained only ~2/3 of the
~16 MB/s dump rate, so the /dev/shm staging backlog grew ~2.2 GiB/min,
filled the node's 256 GiB of RAM in ~1.8 h, and every later OSIRIS HDF5
write failed on ENOSPC (truncated 2 KB dumps, 2 OOM kills, 26 h TIMEOUT
at 16% of tmax). Details in osiris-lpi NOTES.md + dev_docs/
stream-drainer-recommendations.md.

Throughput (the drainer needed ~1.5x; this buys ~20x):
- StreamWriter batches appends and lands them chunk-aligned: one
  resize_dimension + one slice write per batch instead of a gzip
  read-modify-write of the whole ~1 MiB chunk per row (~40x write
  amplification). Measured 23x on the production field shape
  (1765 vs 76 appends/s; the job needed ~55/s per sim).
- Default compression gzip 4 -> 1 (the drainer is compression-bound;
  offline postproc can recompress).
- Discovery is cached: the full MS/ rglob walk, whose cost grows with
  the backlog, runs every rediscover_every polls instead of every poll.
- No thread pool on purpose: h5py serializes all HDF5 calls behind a
  process-global lock, so batching is where the throughput is.

Safety:
- Backlog spill valve (staging mode): past spill_backlog_files/_bytes,
  or under floor_free_bytes on the staging fs, a diagnostic flips to
  mirror-only draining (plain copy to persist MS/, ~10x cheaper), so
  the ramdisk keeps draining no matter what; the NetCDF is caught up
  from the mirror at finalize (mirror consumed + pruned in
  discard_grid_h5 mode). OSIRIS must never see ENOSPC on a dump.
- Corrupt dumps are quarantined (*.h5.bad) and the stream continues;
  previously one bad dump dropped the writer and the watcher re-hit the
  same file every poll, forever. Bookkeeping is now iteration-based
  (recovered from the iter coordinate on resume), not positional, so
  skipped dumps cannot shift slots.

Observability:
- One stats line per stats_every_s (streamed/spilled/backlog/
  quarantined + staging free space) so a smoke can assert the
  steady-state backlog is flat while OSIRIS runs; checking the ramdisk
  after the job proves nothing.
- Repeated identical errors log once per error_log_every occurrences.

API and on-disk schema unchanged (runner.py untouched); all 17 existing
stream tests pass unmodified, plus new coverage for quarantine and the
spill valve in both staging modes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- New raman.py light-wave module and SRS coupling terms in epw.py
- Raman seed/light-wave config in datamodel and helpers
- SRS example config (configs/envelope-2d/srs.yaml) and test
- Document SRS options in lpse2d config docs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The field-save interpolators assumed >=2 transverse cells: interpax.interp2d
returns NaN off a single y-node, and RegularGridInterpolator fills 0 for a
single-node y axis. This blanked all real-space/k-space field artifacts
(fields.xr, k-fields.xr, plots/<field>/*) for ny=1 runs while the series.xr
scalars stayed valid. Add an ny==1 path that interpolates in x only and keeps
the single transverse row, for both the in-solve field saver and the
background-density save.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@physicistphil physicistphil added the duplicate This issue or pull request already exists label Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

duplicate This issue or pull request already exists

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant