Skip to content

Releases

solprop 0.3.0

Choose a tag to compare

@jtravs jtravs released this 18 Nov 23:53
029ea8e

solprop 0.3.0 runs every native simulation in a compiled session of its own and fits the
native backend to the machine it runs on, lets the desktop front end run simulations and
scans on other machines, and makes the Python interfaces consistent, at the cost of many
breaks. It computes in double precision only, has two backends, native and numpy_ref,
and writes output schema 3. Every break is listed first, under "Changed (breaking)", with
how to migrate (ADR-058).

Highlights:

  • Output schema 3: a file or scan written by 0.2.0 is refused wherever it is opened,
    read or resumed, and the refusal names the release that wrote it: read, resume or
    continue it with solprop 0.2.x. Every run's configuration hash changes too, so no run
    started on 0.2.0 continues on 0.3.0.
  • One rule for saved planes, and consistent interfaces
    (SEP 51):
    every route records both ends of its run, n_save=N gives N evenly spaced planes and
    save_planes=[...] lists positions (save_planes=80 is refused with "pass
    n_save=80"). A gas fill needs its pressure wherever a gas is named, loss= becomes
    loss_scale= and alpha=, from_free and prop_free take a RadialGrid in place of
    from_radial and prop_radial, the pulses take phases= and m=, and a
    Simulation's inputs are read-only, changed through with_input or with_mixture.
  • The front end on other machines: the desktop front end runs every simulation and
    scan as a process of its own that outlives the window, on this computer or over SSH on
    a workstation, a Slurm cluster or a rented machine. A Runs window lists every run on
    every machine, and a machine can be set up with solprop from the window
    (Running on other machines).
  • Machine calibration
    (SEP 46):
    solprop calibrate measures how the native backend should thread and plan its
    transforms on this machine, and every later run applies the profile and records it. The
    first native CPU run with no profile for its machine, build and allocation calibrates
    itself first, in about 30 s (SOLPROP_CALIBRATION=use never measures, off ignores
    profiles), and FFTW plans are kept, so each transform shape is measured once per machine
    and build.
  • Faster native runs
    (SEP 45,
    SEP 47):
    the native backend steps every run it admits in a compiled session, Runge–Kutta,
    resumed and dense-output runs and the transverse edge included, with no Python step
    loop, on CPU and GPU builds alike. Idle OpenMP threads spin briefly instead of sleeping
    (benchmark windows took about 0.8 of their time), and on Apple silicon the thread budget
    is the fastest cores, smaller passes thread and SLEEF's vector maths is the default.
  • TDSE on the native backend
    (SEP 49):
    the TDSE response runs on the native CPU build, and backend="auto" selects native
    for it as for every other run. Its atomic provider runs in a worker process of its own,
    so Ctrl-C or a stop flag stops a TDSE run during an atomic call, and solprop no longer
    pins a Pyroraptor revision.
  • More correct results: the zero-dispersion search (modes.base.zdw, the zdw
    statistic and the design tools) no longer reads a gas resonance or the fine structure
    of a narrow feature as a zero, and takes near= to choose among several;
    refractive_index_minus_one keeps its precision in a dilute gas;
    FieldGrid.from_limits refuses ffac below 2.2, where
    the Nyquist bin kept field; and VincettiMode uses the effective-radius coefficient
    Rosa, Melli and Vincetti print, 20 rather than 10, which moves an antiresonant fibre's
    dispersion slightly (results with the old coefficient need solprop 0.2).
  • Retired: single and mixed precision (the precision keyword is gone), the
    experimental mlx backend, the Batch scan executor (use Local or Pool), the
    comparison Adams variants (ComparisonMultistep) and the CF4 coupled-flow sections.
    The backend and translation layers, unstable within 0.x, now build a backend for one
    problem and its run request
    (SEP 52).

Changed (breaking)

Output and resume

  • Output files are written in output schema 3. A file's march snapshot (/march_snapshot) stores its state as (n_modes, n_pol, n_omega) and its engine rows as (n_slots, n_modes, n_pol, n_omega), without the leading axis of one they kept before, and the plane-reference record (RunMetadata.reference) has no basis entry: a plane's basis is derived when it is read. Saved fields, statistics and the rest of the run record are laid out as before. A file or scan written by 0.2.0 (output schema 2) is refused wherever it is opened, read or resumed, and the refusal names the file and the release that wrote it: read, resume or continue it with solprop 0.2.x, and a 0.1.0 file (schema 1) with solprop 0.1.x. Every run's configuration hash also differs from 0.2.0's, since the run record names its record types by module (solprop.output.records.GridRecord) and holds the keywords renamed in this release, so a run or scan started on 0.2.0 is finished on 0.2.x.
  • solprop.output.schema is split into five modules: solprop.output.serialisation
    (serialise, ArrayDigest, StatedInput, recorded_input),
    solprop.output.provenance (BackendRecord, GitProvenance, package_versions),
    solprop.output.records (GridRecord, ModeRecord, StepperRecord, SavedPlane,
    RunOutcome and the other record types), solprop.output.metadata (RunMetadata,
    SCHEMA_VERSION, resolve_metadata_write) and solprop.output.store (Output,
    PlaneStore, FieldRetention and the other save protocols). Import these names from
    solprop.output, which still exports the public ones, or from their new module.
  • A run no longer computes or records its planes' basis identities before its first step (#874): a guide that varies along z paid a mode evaluation at every saved plane, about 0.3 s for 201 planes of a gas-filled capillary with a pressure gradient. A plane's basis is derived when it is read (PlaneBasis.of_run; the record change is output schema 3's), an ordinary mode set's from its modes at the plane's own position, which come from field_plane(..., source=simulation) or from the recipe the output records. A plane of a run from a script read without source= has no basis identity (PlaneBasis.identity may now be None), and same_basis refuses to copy it or apply a declared BasisChange to it, naming source= as the remedy. Recipe.from_output also takes an opened output.
  • A modal basis identity (solprop.coupling.modal_basis_identity) is formed by rounding aeff and neff to 40 significant bits with exact binary arithmetic and hashing their bytes: it costs about a third of what it did and is the same on every platform. Identities therefore differ from those 0.2.0 computed.
  • solprop.output.resume is renamed solprop.output.read_resume_state (also in solprop.output.hdf5), and solprop.sol no longer exports resume: the name read a file's resumable state into a ResumeState and continued nothing, which resume=True on a run or solprop resume does. Replace from solprop.output import resume or from solprop.sol import resume, and resume(path), with read_resume_state; the old name is gone.
  • solprop.driver.run.open_run_record is removed in favour of solprop.driver.run.open_run,
    which opens an output with the same record and also returns its checkpoint interface.
  • MemoryOutput now returns read-only views of its buffers (run["field"], run["z"],
    run["stats/<name>"], the step history and statistics, noise_phase, plane_field): a
    write through one raises ValueError instead of silently changing the stored run. Take
    .copy() of an array to change it.
  • The Output protocol now includes plane_field(index, convention="optics"), which
    solprop.processing uses to read one saved plane: a handler of your own implements it
    (returning shape (n_modes, n_pol, n_omega)) beside fields. CallbackOutput refuses it
    by name. processing.axes.layout reads the shape from the store without reading the
    field, and counts every saved plane.
  • A refused argument to SnapshotController (interval_seconds, max_snapshots) or to
    TDSEArchive (path, outputs) raises the new
    solprop.exceptions.OutputRequestError, a SolpropError and a ValueError, as a
    refused progress or resume raises a DriverError. They raised OutputError (a
    RuntimeError) and bare TypeError or ValueError before. Catch OutputRequestError
    or ValueError in place of OutputError or TypeError around these constructors;
    OutputError still reports a file that cannot be written or read.

Routes and Simulation

  • Every route now follows one rule for the saved planes and the end of a run
    (SEP 51).
    A run marches from initial_z to its length, length or grid.length, and always
    records both ends. n_save=N gives N evenly spaced observations over that interval,
    both ends included, and save_planes=[...] lists positions inside it.
    SimulationSettings gains n_save, its save_planes becomes optional (given neit...
Read more

solprop 0.2.0

Choose a tag to compare

@jtravs jtravs released this 07 Nov 13:03
4b22e5c

solprop 0.2.0 adds a desktop front end, describes every run as a recipe that can be
checked, saved and rerun, and retires the numba backend in favour of the native one,
which is now required. Every break is listed first, under "Changed (breaking)", with how
to migrate (ADR-058).

Highlights:

  • solprop-gui, a desktop front end, experimental in this release and published
    as a third distribution at solprop's own version (pip install "solprop[gui]"): it
    sets up, runs, scans and reads simulations without a script.
  • Recipes, the command line and design solvers
    (SEP 41): a TOML recipe
    for every route, the solprop command, pins that set a radius or a pressure from a
    zero-dispersion wavelength, a dispersive-wave wavelength or a soliton order, chosen
    launch modes, displaced and tilted beams, and parameter metadata for front ends.
  • The native backend is required
    (SEP 43): numba_cpu and the
    numba and pyFFTW dependencies are gone, pip install solprop installs
    solprop-native, and new wheels cover Windows on Arm and x86-64 processors without
    AVX2.
  • Exact resume: a file continues only under an identical configuration hash, as in
    0.1.0, a refusal names the fields that differ (ADR-066), and solprop resume
    continues a run from its file alone.
  • Output schema 2: every field of a recorded dataclass is recorded, a DataPulse
    keeps its phase, large record arrays are stored beside the record's JSON, and a file
    no longer grows with the square of its planes. Read a 0.1.0 file with 0.1.0.
  • Transverse absorbing boundaries
    (SEP 40): every
    free-space run carries a safety ramp by default, a designed real-space collar on
    request, and a boundary monitor that warns or stops when light reaches the edge; that
    edge is now read where the state lives
    (SEP 44), so a device run
    copies no state to check it.
  • MLX performance: compiled stages run a trial 1.5 to 2.6 times faster on an Apple
    GPU, single precision admits the Adams integrator, and single-precision grids are
    sized for MLX's transforms.

Changed (breaking)

The native backend replaces numba

  • The numba_cpu backend, and the numba and pyfftw dependencies, are removed
    (SEP 43, ADR-064).
    backend="numba_cpu" raises BackendError, naming native and numpy_ref. A file
    written by 0.1.0 is refused by its output schema, as every file of another schema is:
    read it with solprop==0.1.0, which is also the rollback. To migrate, pass
    backend="native" (or leave backend="auto"), or numpy_ref for the reference.
  • The settings that only numba_cpu read are gone: SOLPROP_NUMBA_THREADS,
    SOLPROP_NUMBA_REGIME, SOLPROP_FFTW_THREADS, SOLPROP_TRANSVERSE_FFTW_THREADS,
    SOLPROP_PASS_TIMERS, NUMBA_DISABLE_JIT and NUMBA_NUM_THREADS. Drop them from job
    scripts; they now do nothing. SOLPROP_THREADS, SOLPROP_NATIVE_THREADS,
    SOLPROP_FFTW_PLAN, SOLPROP_FFTW_WISDOM and SOLPROP_BLAS_THREADS stay.
  • solprop-native is a required dependency, with no extra: pip install solprop
    installs it, and solprop is supported where it installs
    (SEP 43). On any other platform,
    including Intel macOS, ppc64le, musl and macOS before 14, the installer reports that no
    matching distribution exists; the installation page explains that and how to report a
    platform. The native extra is gone: install solprop, not solprop[native].
  • backend="auto" has no fallback: it is native, and refuses with no native package
    or on a processor below the floor, such as Arm below Neoverse N1. A TDSE response still
    resolves to numpy_ref and single precision to mlx. numpy_ref stays selectable by
    name, as the numerical reference and for the features only it runs. Where auto
    refuses, pass backend="numpy_ref" explicitly.
  • KERNEL_SCHEMA_VERSION and KernelProblem.schema_version are gone, and with them the
    kernel schema number, and solprop_native.BuildInfo.schema_version is now
    source_digest, the SHA-256 of the C++ sources and CMakeLists.txt the extension was
    compiled from: a native run's record names it in backend.settings in place of the
    number. Neither enters a configuration hash or a resume check.
    solprop_native.check_build() takes no problem, and in an editable checkout refuses an
    extension built from other sources than native/, naming uv sync --reinstall-package solprop-native (ADR-068). To migrate, read build_info().source_digest where code
    read schema_version, call check_build() with no argument, and drop
    solprop.translate.KERNEL_SCHEMA_VERSION from imports and checks: solprop-native is
    pinned at solprop's own version, which is now the whole contract.
  • solprop no longer checks a run's memory against a budget before allocating (ADR-065).
    memory_budget=, SOLPROP_MEMORY_BUDGET, estimate_memory, estimate_guide_memory,
    solprop.driver.memory, MemoryBudgetError and MemoryBudgetWarning are gone. A run
    that doesn't fit now fails with MemoryError, or is killed by the kernel or a
    scheduler. Scan workers and TDSE's memory_limit are still bounded; going over that
    limit raises DriverError, and it now covers atomic storage alone. Drop
    memory_budget= from calls and SOLPROP_MEMORY_BUDGET from job scripts; bound memory
    with the scheduler instead, and read Simulation.estimate() for an estimate.

Output files and resume

  • The output schema is 2: a DataPulse records its spectral_phase (below), and every
    field of a recorded dataclass is recorded, at its default too. This release refuses a
    file written by 0.1.0, and every configuration hash moves, so a 0.1.0 run cannot be
    resumed; read such a file with 0.1.0. Code that reads a file with h5py meets these
    changes of layout:

    • /meta/arrays/<name>: a run record's numeric arrays of more than 4096 elements,
      which /meta/json now references (below);
    • /record_arrays/: the same arrays in a scan file, shared between points;
    • /meta/outcome: a Multistep call ledger's predictor orders as [order, count]
      pairs (below);
    • absorbed_power: 5 columns, where it had 4 (below);
    • a top-level recipe group in a run built from a recipe.

    Read files through solprop.output and solprop.processing where possible; they
    handle each of these.

  • resume takes False, True or "interpolated", and any other value is a
    ValueError: a mistyped word used to read as True. A resume continues only on an
    exact configuration hash, as in 0.1.0, and a refusal now names the fields that changed
    (ADR-066). A changed tolerance or integrator, an optmat release, a commit that adds or
    moves a recorded number, and another kind of machine are all refused; a long job has to
    stay on one CPU family to resume, since the record holds numbers computed on the
    machine that built it. An appending HDF5Output holds the file's own record as its
    metadata after a resume, not the one offered. To migrate, pass one of the three
    values; to continue under changed settings, launch a new segment from a saved plane or
    write a new file.

  • An output file no longer grows with the square of its planes. HDF5Output rewrites
    /meta/outcome in place instead of deleting and recreating it at every plane, a saved
    plane builds and validates its run record once instead of nine times, and the step
    driver keeps only the error norms its controller reads, not one per accepted step: a
    Multistep run saved step by step (step_planes=1) costs 19.5 kB a plane at 707 planes
    and 19.3 kB at 3479, where it cost 23.8 kB and 31.5 kB. To get there a Multistep run's
    call ledger (logical_calls and execution.response_calls in the run's outcome,
    /meta/outcome in an HDF5 file) records its predictor orders run-length encoded, as
    [order, count] pairs, where 0.1.0 wrote one integer per accepted step. Code that read
    that list should expand the pairs, or read the record through CallLedger.from_record,
    which gives the orders step by step
    (#80).

  • absorbed_power has a fifth channel, custom, the power a custom_absorber rate
    removes, and its spectral channel is now the collar alone, so a frequency-selective
    loss reads apart from the numerical collar. The statistic's shape goes from (4,) to
    (5,); the first four channels keep their positions and their sum stays -dE/dz.
    Absorber.user_rate holds the user's share of spectral_rate, which stays the total,
    so the operator does not move (#231). Code that read absorbed_power[..., spectral]
    as all the spectral loss under a custom_absorber should add the new custom column
    (index 4).

Free-space transverse boundaries

  • A free-space run's transverse boundary is a SpatialAbsorber
    (SEP 40), given as
    SimulationSettings.spatial_absorber or the spatial_absorber keyword of
    prop_radial, prop_free, prop_planar and their Simulation.from_* builders. By
    default every radial, Cartesian, line and planar run carries a designed safety ramp on
    its outer 10 % of transverse wavenumbers, folded exactly into the operator, radial and
    planar included; spatial_absorber=False removes ...
Read more

solprop 0.1.0

Choose a tag to compare

@jtravs jtravs released this 01 Nov 07:35
5eb9c77

The first public release of solprop and of its compiled backend, solprop-native,
which is released at the same version. Releases are 0.x for now: a minor release may
break the API, the output schema or the resume key, and lists every such break under
"Changed (breaking)" (ADR-058).

Added

  • Propagation equations. The unidirectional pulse propagation equation (UPPE) and
    the generalised nonlinear Schrödinger equation (GNLSE), for the real electric field or
    a complex envelope, in waveguides and in free space.
  • Waveguides.
    • Gas-filled hollow capillaries, with resolved modes (the default) or mode-averaged
      (mode_average=True on prop_capillary and prop_waveguide). Tapers,
      pressure gradients and gas flow are supported, as is a coherent wall that couples
      the modes.
    • Antiresonant hollow-core fibres (the Vincetti model).
    • Step-index, graded-index, rectangular and planar guides.
    • Modes tabulated from arrays or from external mode solvers.
    • A general modal route for any guide that supplies its own electric and magnetic
      mode fields, including guides that change along their length.
  • Free space. Radial, Cartesian and line grids with the exact (non-paraxial)
    diffraction operator; planar guides can be resolved across their width. Launches,
    shot noise, energies, fluences and intensities measure the medium's axial Poynting
    flux at the plane, so a requested energy is the physical energy there
    (SEP 34).
    Far-field and aperture statistics.
  • Media and responses.
    • Gases, gas mixtures, glasses and metals from the optmat
      material library.
    • Kerr and third-harmonic nonlinearity, including a dispersive electronic response.
    • Photoionisation with atomic and molecular rate models, and plasma on field and
      envelope grids.
    • Rotational and vibrational Raman responses, scalar and vector.
    • Quantum shot noise, on by default
      (SEP 37):
      one photon per mode with a random phase, added to the input field (noise="input")
      and recorded with its seed. A resumed run draws the same realisation again, and a
      scan draws each point's from a seed of its own.
      noise=None switches it off and noise="modified" keeps the realisation apart
      from the field. The Cartesian, line and hybrid planar grids, slab mode sets, the
      TDSE response and single precision carry none by default and record why. Spectral
      densities can subtract the one-photon floor (noise_floor="subtract").
    • An optional helium TDSE atomic response for research. Its native backend is not
      yet publicly available.
  • Integrators and their defaults.
    • Two adaptive integrators: a low-call exponential Adams method (Multistep()) and
      an embedded interaction-picture Runge–Kutta pair (RungeKutta()), with a
      low-storage Runge–Kutta variant and fixed schedules. The linear propagation is
      integrated exactly.
    • Each interface has its own default integrator and tolerances, chosen by
      measurement: RungeKutta() for prop_gnlse (rtol 1e-7), prop_waveguide (1e-5)
      and prop_free (1e-4); Multistep() for prop_capillary (1e-6), prop_radial
      and prop_planar (1e-5). The
      choosing-a-stepper page
      tabulates them and says when to choose another.
    • Multistep(carrier_frame="scalar"), an opt-in carrier frame for the Adams history
      that is the fastest method measured on dense, multimode, Raman-active runs such as
      a comb-to-continuum capillary
      (SEP 35).
    • Backends agree with each other to bounded, rounding-level differences.
  • Time-window boundaries
    (SEP 21).
    Every interface absorbs at the edges of the time window and of the spectrum. The
    temporal collar lies outside the requested window: the grid widens to
    trange / (1 - 2 collar_fraction) (collar_fraction 0.1), so the whole request is
    flat, and transform sizes default to fft_size="auto", the cheapest 2-3-5-smooth
    size up to the next power of two. absorber=False switches the soft collars off and
    keeps the same grid. A boundary monitor, on by default (boundary_check= on every
    prop_* function and Simulation.from_* constructor), warns (BoundaryWarning) and
    then stops the run (BoundaryError) when light above the shot-noise floor reaches
    the window's edge, and records the energy the collar removed.
  • Coupling simulations
    (SEP 31).
    solprop.coupling hands a field from one simulation to the next. A FieldPlane
    holds a coherent field at one plane with its reference and provenance; couple maps
    it into another simulation's InputPort under a declared CouplingModel and returns
    an energy report; transport carries a free-space plane through lenses, apertures,
    masks, mirrors, interfaces and spectral filters; Outcouple synthesises a guide's
    exit field. An incident beam launches into a general modal guide with the share of
    its energy that the guide captures. Each step is admitted against an error budget and
    refuses what it cannot certify.
  • Backends.
    • numpy_ref, the NumPy reference, against which every other backend is tested
      within derived error bounds, and numba_cpu, compiled with Numba. The backends come
      in tiers: new features land on numpy_ref and native; numba_cpu is mature and
      takes a new feature only where its design says so; mlx is experimental.
    • The optional native backend,
      the solprop-native distribution: C++20 kernels on Kokkos for multi-core CPUs
      (OpenMP) and for NVIDIA and AMD GPUs (CUDA, HIP). pip install "solprop[native]"
      installs a precompiled CPU wheel on macOS on Apple silicon, Linux on x86-64 and
      aarch64, and Windows on x86-64. backend="auto" selects it when installed, except
      on a processor below the wheel's CPU baseline, where it uses numba_cpu and warns
      once (SEP 29).
      On the native backend the exponential Adams march runs in compiled code, many steps
      per call
      (SEP 39).
    • One thread budget per process, which every pool of both CPU backends stays within:
      the smallest of the CPU affinity mask, the physical cores that hold it, the
      control group's CPU quota and SOLPROP_THREADS (ADR-052).
    • Single precision
      (SEP 33):
      precision="single" holds the state, the transforms and the responses in binary32,
      with the controller in binary64. numpy_ref runs it as the reference, and the mlx
      backend (pip install "solprop[mlx]") on the Apple GPU. A
      survival record
      states which problems keep their accuracy.
  • Running and recording.
    • HDF5 output with periodic snapshots and provenance, and resuming an interrupted
      run, which continues the Adams history.
    • A summary at the start of every run of what it is about to compute (the code, the
      machine, the backend, the grid, the medium, the responses, the noise, the
      integrator, the absorber and the output), and at its end the wall time, CPU time
      and peak memory. Each execution records its machine and what it spent in the file.
    • Statistics along the propagation.
    • Parameter scans on process pools and Slurm clusters.
  • Analysis and tools.
    • Processing and plotting. Spectral ranges and band-pass windows are given in the
      units of any spectral axis (spec_range, Passband).
    • Ensemble coherence by spectral band, and replay of a run's accepted steps
      (FixedSchedule.from_history), to compare two precisions or backends on one
      discretisation.
    • Soliton and dispersive-wave design tools.
    • Pulse compressors: Taylor phase, chirped mirrors, materials, gratings and prisms.
    • An interactive session through import solprop.sol as sol.
  • Performance tracking for developers
    (SEP 38):
    python -m benchmarks track times about a dozen representative workloads, with each
    interface's defaults, at one thread and at the production width, and records them by
    hardware; python -m benchmarks ab compares two builds.
  • Documentation. An introduction with executable lessons, a user guide, a
    long-form primer on the physics and numerics, focused tutorials and worked examples,
    the API reference, and a developer's guide with the design records (SEPs and ADRs).

Changed (breaking)

Nothing was released before 0.1.0. For scripts and files from a development checkout:

  • Output files of development versions are not read. The output schema restarts at
    1 (solprop.output.SCHEMA_VERSION); keep the development install that wrote such a
    file to read it. The kernel contract's version is KERNEL_SCHEMA_VERSION, separate
    from it.
  • Defaults changed late in development, so a development script's results change:
    shot noise is on; each interface takes its own integrator and tolerances; the
    temporal collar lies outside the requested window, which widens the grid
    (`collar_pla...
Read more