Modelly 1.4.0

The scale-out release: split runs across processes, a GPU backend, differentiable models,
live data and assimilation, fields and ODEs coupled to agents, emulators and history matching,
synthetic populations, a static export that runs a model in the browser, and Studio II. No
playbook is slower than 1.3.3 and every output is identical; the event-heavy wolf-sheep example
runs at 0.95, event scheduling at 0.70 and column data collection at 0.47 of 1.3.3's time
(docs/performance.md). Hosted CI now runs only the Release and docs workflows; every other
check runs locally (scripts/release-checks.sh), on Linux x86-64.

Added

• Per-agent random streams (provisional): with agent_streams = True on a model class,
  agent.random and agent.rng are generators of the agent's own, seeded from the model's
  seed and Agent.stream_key() (unique_id by default). An agent draws the same numbers
  whatever the activation order or the population, and checkpoints continue every stream.
  Off by default; models without the flag draw exactly as before.

• modelly.distributed.partition (provisional): grid_blocks(rows, cols).layout(grid) cuts a
  2-D grid into balanced blocks and gives each cell's owner and each block's halo, on plain
  grids and tori.

• Split runs (modelly.distributed, provisional): a model that declares
  partitionable = Partitionable(space=..., phases=...) and uses per-agent streams is built
  split across workers with Cluster(backend=...).build(Model, scenario, partition), with ghost
  agents exchanged at partition borders before a phase that looks across them and agents
  migrating with their streams after a phase that moves them. Backends local, mpi
  (mpi4py, new distributed extra), dask and ray (imported only when used). Results equal
  one process for the same seed, byte for byte; a model that breaks the contract (model-level
  draws in a phase, Column agents, own events, moves into cells with a capacity, agent
  references in attributes, births keyed on unique_id) is refused with the reason, and
  owns() lets a model build only its partition's agents.

• Graph and k-d partitioners (provisional): partition.graph(parts) splits a Network by
  breadth-first banding and greedy boundary refinement, partition.kd(parts) a
  ContinuousSpace by recursive bisection at the population quantile; both deterministic,
  pure numpy, with halos by hops or by distance.

• Distributed collectors and checkpoints (provisional): Sum, Min and Max reporters reduced
  across partitions (collect()), agent snapshot()s in stream-key order, and
  checkpoint(directory) writing one file per partition plus an index.json with digests,
  which Cluster.restore continues draw for draw.

• Partition-independence check (provisional): check_independence(Model, scenario, partitions, steps) runs one process and each split and raises at the first differing reporter or agent
  state; the test suite runs it on a grid, a network and a continuous model at 1, 2, 4 and 16
  partitions, and on MPI with 2 and 4 ranks. benchmarks/distributed/weak_scaling.py measures
  weak scaling, with the guide "From laptop to cluster" (docs/distributed.md).

• modelly.stream (provisional): ReplaySource, CSVSource (a growing file) and
  HTTPSource hand out observations once each, in time order; ObservationOperator reads the
  same quantities from a model, with residuals and a Gaussian log-likelihood.

• modelly.stream.Ensemble (provisional): n replicates of one scenario with spawned seeds,
  advanced in lock step in this process, with starting values drawn from a prior; band and
  forecast give the (weighted) mean and quantiles of an operator now and ahead as a
  long-format DataFrame (time, quantity, mean, q0.05, ...), for plots and live views.

• modelly.stream.ParticleFilter (provisional): the bootstrap particle filter (Gordon et al.
  1993) over an ensemble, with the effective sample size tracked and systematic resampling
  (Douc & Cappé 2005) below a threshold; duplicates are checkpoint copies reseeded from derived
  keys, so a run is bit-identical for a seed with the compiled or the pure-Python core. The new
  nowcasting tutorial (docs/assimilation.md) tracks a synthetic truth with it.

• modelly.stream.EnsembleKalmanFilter (provisional): the stochastic ensemble Kalman filter
  with perturbed observations (Evensen 1994, 2003) on named model attributes
  (state/parameter augmentation), with optional bounds; deterministic for a seed.

• modelly.stream.MQTTSource and KafkaSource (provisional): sources reading JSON records from
  an MQTT or a Kafka topic, through the new optional extras mqtt (paho-mqtt) and kafka
  (confluent-kafka), or through a client passed in.

• population: modelly.population, synthetic populations from aggregate tables: ipf and
  trs (truncate, replicate, sample) rounding in synthesize_population, marginal_errors
  to check the fit, anneal (combinatorial optimisation of whole records by simulated
  annealing), assign_households from a household-size table, locate_households and
  assign_places (nearest school or workplace with room, within a distance limit), and
  contact_network (household, school and work layers of a MultilayerNetwork). No bundled
  personal data; modelly.space.geo.ipf and synthesize_population keep working.

• zoo: age_seir.model.scenario_from_population starts the age-structured SEIR model from a
  synthetic population (group sizes and contact matrix); the default scenario is unchanged.

• geo: modelly.space.geo.samples, small real samples of Burlington, Vermont, bundled for
  offline use with their licences in NOTICE.md (OpenStreetMap streets, ODbL; ESA WorldCover
  land cover, CC BY 4.0; census tracts, public domain), and scripts/geo_data.py to download
  the full sources, verify their SHA-256 and rebuild the samples.

• zoo: road_traffic, traffic assignment on the OpenStreetMap streets, validated against the
  exact edge betweenness.

• zoo: habitat_selection, Metropolis dispersal over the WorldCover raster, validated against
  its stationary law.

• zoo: schelling_regions, Schelling segregation on the census tracts by contiguity.

• modelly.couple: multi-scale coupling. Field makes a grid property layer diffuse, drift and
  decay on its own sub-steps (finite volumes, explicit with a stability check or implicit;
  closed, absorbing or periodic boundaries; mass conserved to round-off, with a budget of what
  was added, left and decayed); ODE and StockFlow integrate continuous components with
  scipy's solve_ivp; Coupling and Phase run them with the agents' step phases, checked at
  build time for reads with no writer, two writers and cycles inside a phase. Coupled models
  checkpoint and resume bit for bit. Guide: docs/coupling.md.

• Two coupled zoo models: nutrient_foragers (foragers on a diffusing nutrient field,
  validated against the carrying capacity of the energy balance) and hospital_epidemic (SIR
  agents coupled to a hospital-capacity stock-flow ODE, validated against the final-size
  relation and the hospital's exact linear solution), both with mass-balance tests.

• modelly.emulate emulators (provisional): GaussianEmulator (one Gaussian process per
  output, fitted with restarts, with covariances and conditioning) and NeuralEmulator (a
  numpy ensemble of small networks), fitted to unit-cube points or to an experiment's runs
  (from_results), and usable in place of the model in modelly.sensitivity; Simulation
  runs a model through the experiment executors (or a function) for several outputs, each
  reduced by a statistic, optionally with common random numbers. GaussianProcess.fit takes
  restarts=True and the process gains covariance, condition and noise_variance; the
  default fit is unchanged, bit for bit.

• modelly.emulate.ActiveLearning (provisional): designs that run next where the emulator is
  least certain (integrated variance reduction, or the largest variance), chosen in batches by
  conditioning on each pick, seeded and resumable. Against Latin hypercubes with the same
  emulator (benchmarks/emulate/active_vs_lhs.py), it saves a factor of 1.6 to 2.7
  of the runs on the zoo's Bass diffusion (2 or more at 20, 40 and 50+ runs, 1.3 at 30), but
  only 1.1 to 1.4 on Branin below 80 runs and 0.8 to 1.4 on Ishigami: the roadmap's factor of 2 holds on
  one of the two kinds of benchmark (table in docs/emulators.md).

• modelly.emulate.HistoryMatch (provisional): history matching in waves, with the
  implausibility |z - E[f(x)]| / sqrt(Var_emulator + Var_obs + Var_discrepancy), a cutoff of
  3, NROY tracked on a fixed sample and emulators refitted inside it every wave. On the zoo's
  age-structured SEIR (truth at the default parameters, three outputs), four waves of 20 runs
  leave 17 %, 4.4 %, 3.1 % and 2.5 % of the box, the truth inside every time. Guide:
  docs/emulators.md.

• gpu: modelly.gpu runs column phases (DeviceTable), cohort tables (DeviceCohortTable,
  binomial thinning included) and cell-space operators (DeviceCellIndex: counts, neighbour
  sums, random-neighbour moves) on numpy, CuPy or JAX, chosen by get_backend or
  MODELLY_BACKEND with a numpy fallback; PhiloxStream, a Philox4x32-10 stream keyed by
  (seed, stream), draws the same numbers on every backend (Random123 known-answer vectors);
  plan_memory / run_batched run populations larger than the GPU in batches, bit for bit;
  modelly-gpu variants of the grid epidemic and insect cohorts playbooks. On a laptop
  RTX 5050 the grid epidemic at 10 million agents keeps the CPU run's integer state bit for bit
  and runs its 60 steps 34x faster than the CPU column variant (docs/performance.md).
  Extras gpu (CuPy) and diff (JAX); import modelly loads neither.

• diff: modelly.diff simulates column phases traced by JAX (DiffModel, @phase with a
  differentiable flag) and differentiates their outputs with respect to parameters:
  pathwise normal and uniform draws, Bernoulli choices through straight-through, Gumbel-softmax
  or score-function estimators (each warns GradientApproximationWarning), check_gradients
  against finite differences, and GradientCalibration (Adam on modelly.calibrate priors and
  targets, deterministic for a seed). Differentiable twins of the Bass diffusion and
  Bouchaud-Mezard zoo models recover known parameters; docs/differentiable.md says when the
  gradients mean something.

• quality: partition independence in the determinism suite. The three reference models of split
  runs (grid_epidemic, network_sir, continuous_sir) are golden examples, and
  tests/determinism/test_determinism_partition.py splits each from the golden seed (2, 4 and
  16 partitions) and checks every output equals the single-process run, so a split run on any
  platform gives the golden digest. A new mpi workflow runs the distributed tests under Open
  MPI on Ubuntu and macOS and under MS-MPI on Windows (2 ranks only); test_mpi.py uses
  mpiexec on Windows and reads the rank counts from MODELLY_MPI_RANKS.

• provenance: the language-model response cache travels with the study. modelly run exports
  the replies of the model's modelly.llm.Clients to llm-responses.jsonl.gz and records it in
  the manifest (llm_cache, an optional field of manifest format 1); modelly bundle packs it
  and refuses a run whose replies changed; modelly verify reruns inside the new
  modelly.llm.replay.replaying(path) (provisional), where every client answers from the
  recorded replies only, so a bundle verifies with no network and no API key. Runs without a
  client write the same manifest and bundle as before.

• plugins: Modelly now reads three more entry-point groups. modelly.samplers adds design kinds
  ([design] kind = "name" in an experiment file, designs.design_factory and design_kinds,
  provisional); modelly.lint_rules adds rules to modelly lint (an object with a Rule and a
  check(tree), modelly.lint.plugins, provisional); modelly.studio_views adds Studio tabs.
  A rule or view that fails to load or reuses a built-in code is skipped with a warning. A
  complete third-party package with one entry point in every group is in
  examples/plugins/modelly-example-plugin, tested in tests/test_example_plugin.py.

• modelly.calibrate.ExternalBayesOpt (provisional): Bayesian optimisation by an outside
  library over the same unit box and common random numbers as BayesOpt, through any optimiser
  with ask() and tell(point, value); the built-in backend is optuna (TPE seeded from the
  calibration's seed, or any optuna sampler), from the new optional extra calibrate and
  imported only when the search starts. Also method = "external_bayes_opt" in calibration
  files.

• calibrate: a zoo model calibrated end to end. ABC-SMC recovers the Bass diffusion model's
  innovation and imitation inside their 95 % intervals on three seeds, from priors ten times
  wider (tests/calibrate/test_zoo_end_to_end.py, "A zoo model end to end" in
  docs/calibration.md).

• sensitivity: Morris stability across seeds on zoo models. The whole ranking of the Bass
  diffusion model's three parameters, and the top parameter of the age-structured SEIR model,
  are the same on five seeds (tests/sensitivity/test_morris_stability.py); the guide says
  which parameters swap places and why.

• docs: design reviews of calibration (E1), sensitivity analysis (E2) and language-model agents
  (E7), each what was decided, what else was considered and why (docs/design/).

• quality: WebAssembly in the release matrix. The release workflow's new pyodide job installs
  the pure wheel with micropip in Pyodide under node, imports modelly, checks the pure-Python
  twins are in use and runs the Boltzmann wealth example (scripts/pyodide-smoke.mjs); the
  PyPI upload waits for it. In Pyodide import modelly needs Pyodide's sqlite3 package, and
  importing any bundled example needs networkx.

• quality: SLURM and PBS job scripts run for real. test_cluster_scripts.py puts stand-in
  sbatch and qsub executables on PATH that run the generated job script in bash once per
  array element, so every element runs in a fresh shell and interpreter and must give the serial
  numbers; the new platforms workflow's slurm job submits the same array to a single-node
  SLURM cluster on the runner (MODELLY_REAL_SCHEDULER=slurm).

• quality: a bundle made on Linux verified on macOS and Windows. The platforms workflow runs
  and bundles three studies on Ubuntu (Boltzmann wealth, Schelling, and the language-model
  opinions example, whose replies travel in the bundle), and macOS and Windows jobs unpack each
  and run modelly verify.

• release: .zenodo.json at the repository root, so the GitHub-Zenodo integration archives each
  GitHub release with the same title, licence, keywords and creators as CITATION.cff (no
  contact data; the version comes from the tag), checked by tests/test_zenodo.py.

• lint: MDL011 (a scipy.stats.qmc engine, a .rvs() draw or a SALib sampler without a seed),
  MDL013 (a mutable default in a Scenario, shared by every scenario of the class) and MDL014
  (a clock, timer or OS entropy read in a calibration Target); --fix seeds a sampler in a
  model or agent method from self.rng. 21 rules, five with a fix.

• modelly lint --lsp: the linter as a language server over stdio (standard library only),
  publishing findings as diagnostics on open, change and save, with the safe fix and a
  # noqa comment as quick fixes; settings for Neovim, Helix, VS Code and PyCharm in
  docs/lint.md.

• .pre-commit-hooks.yaml: modelly-lint and modelly-lint-fix hooks, so a project's
  pre-commit configuration runs modelly lint (or its safe fixes) on the staged files.

• modelly new --template geo (residents of a town's districts in a GeoSpace) and
  --template llm (citizens deciding through modelly.llm, offline by default); both pass
  their determinism test and modelly lint as generated.

• DataCollector.to_xarray (and modelly.data.xarray_export.to_xarray): model reporters as an
  xarray.Dataset over time, arrays such as a grid stacked into (time, ...), agent
  reporters as a (time, AgentID) panel; xarray is the new optional xarray extra.

• docs: From R and Julia, reading Modelly's Parquet, DuckDB and netCDF results with arrow,
  duckdb, Parquet2.jl and DuckDB.jl, and running Modelly from R (reticulate) and Julia
  (PythonCall); the Python side of every snippet runs in the test suite.

• modelly netlogo (python -m modelly.netlogo): translate a NetLogo model (.nlogo) into a
  runnable, seeded, lint-clean Modelly model for a documented subset (turtles, patches, breeds,
  ask, set, if/ifelse, repeat, create-/hatch/sprout, movement, neighbors,
  one-of, random, globals; sliders and switches become Scenario fields). Lines outside the
  subset stay as # NETLOGO comments and are listed with their line numbers. Ships Fire and
  Wolf Sheep Predation sources written from the published descriptions; the translated Fire
  model shows the percolation threshold near 59% density. See docs/netlogo.md.

• Strict type checking: [tool.mypy] in pyproject.toml runs mypy in strict mode on batch,
  bench, calibrate, dev, experiments, lint, llm, netlogo, new, profile, provenance, scenarios,
  sensitivity, timetravel, the data exports and the top-level modules (156 files); a new types
  stage of scripts/ci-local.sh and python -m modelly.dev gate runs it with installed mypy,
  else the pinned mypy through uvx, and is skipped with a message when neither is available.

• web: modelly export-web model.py:Model --out site/ writes a static site that runs the
  model in the browser with Pyodide (pinned 314.0.7, from its CDN): Studio's parameter panel,
  run controls, a space view and a reporter plot, the model in a Web Worker, Modelly as a
  trimmed pure-Python wheel and only the packages the model imports; every example and zoo
  model runs from its export (tests/web/test_browser.py, CI job browser), and the
  gallery embeds three live models. A first visit transfers 14 MB (Pyodide, numpy, pandas)
  and is ready in 8 s on a 50 Mbit/s line, 21 s on DevTools "Fast 4G"
  (tools/web/measure_load.py). Guide: docs/browser.md.

• native: the C modules build for WebAssembly with Emscripten (pyodide build,
  scripts/build-wasm-wheel.sh); modelly export-web --wheel ships that wheel.

• Memory-mapped columns (modelly.core.storage): model.columns.storage = MemoryMapped(dir)
  or CohortTable(rng, storage=MemoryMapped(dir), ...) keeps a table's columns in
  numpy.memmap files, grown in place and deleted by close(), a with block or garbage
  collection; checkpoints carry the rows and load them into a new folder.
  benchmarks/large_columns.py runs a cohort population of any size on either storage.

• modelly bench --against VERSION: every Modelly playbook variant in this checkout and in a
  release built from its tag (same compiler and dependencies), alternating, results checked
  identical, with the median and interquartile range of the per-round CPU ratios; a variant the release cannot run (one added since it) is
  left out and listed, so the comparison covers what both sides have.

• benchmarks/pgo.py: the compiled modules built with GCC profile-guided optimisation
  (trained on the playbooks, benchmarks/pgo_training.py) and timed against -O3 on every
  playbook. It measured no faster, so the build does not use it (docs/performance.md).

• Studio Ensemble tab (provisional): replicates of the parameter panel's values, seeds spawned
  from the panel's seed, and the mean, median and 5 to 95 % band of every numeric model
  reporter across them step by step, as modelly.stream band frames (CSV download).

• Studio Assimilation tab (provisional): modelly studio --observations observed.csv runs a
  particle filter or an ensemble Kalman filter over the observations and shows the filtered
  bands with the data, the effective sample size and every replicate's weight over time, or
  the adjusted parameters' mean and spread over time.

• Studio Experiments tab (provisional): --results DIR browses an experiment's results store:
  one row per run, filters by parameter (ranges or values), an output against a parameter and
  the selected runs step by step; modelly.studio.browser.ExperimentBrowser from Python.

• Studio Analysis tab (provisional): --analysis file.py:function (or session.analyses)
  shows calibration posteriors (marginals, pairs, fits, HDI summary) and sensitivity results
  (the method's plot, the ranking).

• Studio Map tab (provisional): the space drawn by WebGL in the browser (agents, network nodes
  and edges, GeoSpace region outlines), with pan and zoom, falling back to Canvas 2D without
  WebGL; the renderer ships inside Modelly. 100 000 agents and a 100 000-edge network draw in
  headless Chromium, Firefox and WebKit on Linux; above 10 000 elements the Model tab leaves the
  space to it.

• Studio session script (provisional): Export → Session script downloads a Python script that
  rebuilds the live run (new runs, steps, replays with changed values and seeds), checked to
  give the same model state on six bundled examples.

• Studio bundle export (provisional): Export → Bundle reruns the live run with modelly run's
  recorder and packs it as a modelly bundle zip that modelly verify checks.

• python -m modelly.studio.assets [--fetch] (provisional): lists and downloads the front-end
  files Solara's page needs into its cache, so Studio runs offline; modelly studio warns when
  some are missing. The browser checks, run locally before a release one browser at a time
  (MODELLY_BROWSER=chromium|firefox|webkit), serve the page with the CDN unreachable.

Changed

• The event list's heap is compiled (modelly/_native/fastevents.c): the same ordering (time,
  then priority, then insertion order), the same heap array, a pure-Python twin, and
  checkpoints that load in either; time advance no longer pops and re-pushes the first future
  event, and schedule_event builds one weak reference instead of two.
• The DataCollector's attribute reporters, AgentSet.get and agg read Column
  attributes from their table in C instead of calling the Python getter per agent; the rows
  are identical.
• docs: the build fails on warnings. scripts/build-docs-version.sh passes -W --keep-going by
  default (SPHINXOPTS= builds an old tag that warns) and the docs workflow sets it. The build
  was made warning-free first: a docstring formula read as a substitution, ambiguous Grid and
  Observation type references from imports only made for type checking, and the "no lexer"
  warnings of the text-notebook tutorials, which appear only when they are not executed and are
  filtered in conf.py.

Deprecated

• modelly.space.geo.ipf and modelly.space.geo.synthesize_population (also through
  modelly.space.geo.population, with its constants) warn with a ModellyDeprecationWarning
  and go in 2.0: import them from modelly.population. They are the same functions, so results
  do not change. docs/deprecations.md gains "Announced for 2.0", every change 2.0 makes with
  what to use today and when it starts to warn.

Fixed

• Modelly imports and runs under Pyodide: the profiler falls back to process_time where
  thread_time is missing, the LLM client sends calls one at a time where threads cannot
  start, synthesize_population passes intp counts to np.repeat (32-bit numpy), and
  modelly.examples imports each example on first use instead of all of them.