Skip to content

Optimise GM99 multi-resident establishment (issue #34)#35

Merged
dfalster merged 3 commits into
masterfrom
optimise-gm99-estab
Jun 29, 2026
Merged

Optimise GM99 multi-resident establishment (issue #34)#35
dfalster merged 3 commits into
masterfrom
optimise-gm99-estab

Conversation

@dfalster

@dfalster dfalster commented Jun 29, 2026

Copy link
Copy Markdown
Member

Summary

Fixes the exponential-in-species-count slowdown in gm99_equilibrium() reported in #34. Two changes to src/GM99.cpp:

  • Tighter Poisson truncation in estab(): Kmax goes from N + 10·√(N+1) + 20 to N + 6·√(N+1) + 6. Tail probability beyond the new cutoff is < 1e-10, accuracy is unchanged, but each dimension of the Cartesian odometer has ~2× fewer terms (e.g. N=1: 31 → 16 terms).

  • Newton's method replaces the fixed-point iteration N_i ← N_i · W_i in gm99_equilibrium() for the multi-resident case. Newton solves log W_i(N) = 0 in log-N space with a numerical Jacobian (forward differences). It converges quadratically so only ~10–30 steps are needed vs. thousands for the old fixed-point, which could oscillate slowly near equilibrium. Competitive exclusion is handled by removing residents with near-zero density and negative log fitness from the active Jacobian subsystem.

Before/after benchmarks (alpha=7, beta=15, R=1)

scenario before after speedup
2-resident equilibrium ~3.5 ms 0.11 ms >30×
3-resident equilibrium ~1100 ms 4.7 ms >230×
assembler step at 3 residents ~6000 ms/step <0.1 ms/step >60000×

The 3-resident target from the issue (< 100 ms) is met with room to spare (4.7 ms).

Test plan

  • All tests pass in the current repo state (note: test suite is under active modernisation in a parallel branch — counts may change)
  • test-harness-gm99.R: all oracle tests pass (W_m(m)=1, closed-form g, CSS/branching at αR=4.5/7, αR·βR scaling)
  • test-assembly.R: all assembly tests pass
  • Results numerically identical within existing tolerances (logW < 1e-12 at equilibrium for validated 2- and 3-resident communities)

Closes #34.

🤖 Generated with Claude Code

dfalster and others added 3 commits June 29, 2026 10:16
Two changes to src/GM99.cpp:

1. Tighter Poisson truncation in estab(): reduce Kmax from
   N + 10*sqrt(N+1) + 20 to N + 6*sqrt(N+1) + 6.  The tail beyond the
   new cutoff is < 1e-10, so accuracy is unchanged but each dimension of
   the Cartesian odometer has ~2x fewer terms (e.g. N=1: 31 -> 16).

2. Replace the fixed-point iteration N_i <- N_i * W_i (up to 10000 steps,
   absolute tolerance) in gm99_equilibrium() with Newton's method on
   log W_i(N) = 0 in log-N space, using a numerical Jacobian computed by
   forward differences.  Newton converges quadratically so only ~10-30
   steps are needed vs thousands for the old fixed-point, which could
   oscillate slowly for multi-resident systems.  Competitive exclusion is
   handled by removing near-zero residents (N < 1e-8, log W < 0) from the
   active Jacobian system.

Before/after benchmarks (alpha=7, beta=15, R=1):
  2-resident equilibrium: ~3.5 ms -> 0.11 ms  (>30x)
  3-resident equilibrium: ~1100 ms -> 4.7 ms   (>230x, target was <100 ms)
  assembler step at 3 residents: ~6000 ms -> <0.1 ms (>60000x)

All 218 tests pass unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adapt the plant-family build conventions to regnans:

- Makefile: adapted from plant/Makefile, dropping RcppR6/vignettes/
  website targets (regnans uses plain Rcpp::compileAttributes and has
  no vignettes or pkgdown). Targets: compile, attributes, roxygen,
  test, install, build, check, clean.
- .github/workflows/R-CMD-check.yaml: adapted from plant-dev2's modern
  r-lib/actions workflow (plant itself only has legacy Travis/AppVeyor).
  Runs R CMD check on Windows + macOS.
- DESCRIPTION: add Remotes (traitecoevo/plant, smbache/loggr) so CI can
  install the GitHub-only dependencies.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dfalster dfalster merged commit 44652ff into master Jun 29, 2026
0 of 2 checks passed
@dfalster dfalster deleted the optimise-gm99-estab branch June 29, 2026 00:44
dfalster added a commit that referenced this pull request Jun 29, 2026
Brings `R CMD check` to **Status: OK** (was 6 WARNINGs + 3 NOTEs, plus
an install failure and a CI vignette-build error on macOS).

### Fixes
- **Install failure** — `man/harness_gm99.Rd` reworded so R 4.6's
`parse_Rd` no longer reads apostrophes (`x'`, `paper's`, `Daniel's`) as
quote delimiters (was: *"unexpected end of input"*).
- **Latent runtime bug** — dropped a stray `ctrl = ctrl` arg in
`demography_solve_equilibrium_solve()` (unused-argument byte-compile
note).
- **NAMESPACE / dependencies** — added `@importFrom` for
stats/utils/ggplot2/dplyr/tibble/purrr + `utils::globalVariables()` for
NSE columns; moved `dplyr`/`tidyr`/`mlr3learners` from Depends to
Imports; declared
`bbotk`/`data.table`/`loggr`/`mlr3`/`mlr3mbo`/`paradox`; dropped unused
`progress`; `requireNamespace(mlr3learners)` instead of `require()`.
- **Documentation** — synced `@param` blocks with signatures
(`community_start`, `community_solve_singularity_1D`,
`community_fitness_landscape`); described empty `@param ...`; made
`community_fitness_landscape_bayesopt` internal; fixed broken
`\link{assembler_stochastic_naive}`.
- **Stale generated files** — regenerated `RcppExports.R` +
`gm99_equilibrium.Rd` to match the committed Newton `GM99.cpp`
(`200L`/`1e-10`). Master currently still has the old `10000L`/`1e-12` in
these generated files, so this also clears the codoc mismatch now
present on master.
- **DESCRIPTION** — complete-sentence `Description`; removed `LazyData`
(no `data/`).
- **.Rbuildignore** — `.claude`, `.github`, `.plant-interface-version`,
`AGENTS.md`, dev `Makefile` (GNU-extensions warning).

### Vignettes → staging
The three WIP vignettes (`assembly_fitmax`, `assembly_stochastic`,
`solving_attractors`) ran the full plant SCM on every build (the source
of the macOS CI `creating vignettes ... ERROR`). Moved out to
`overstorey_staging/` as `.qmd`.

### Not touched
`loggr` is left as-is (Imports + `Remotes: smbache/loggr`), matching
`plant`, which uses it and passes on Windows. The earlier Windows
`pkgdepends` resolution error on #35 looks like a transient/knock-on
solve failure rather than a real loggr problem.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[speed] Optimise GM99 multi-resident establishment (Poisson sum is exponential in species count)

1 participant