Skip to content

History / R Python differences

Revisions

  • Hide YAML frontmatter from wiki renderer via HTML comment wrap GitHub's Gollum wiki renderer doesn't strip YAML frontmatter — it shows the title/slug/summary/etc. block as literal text at the top of each rendered page. Wrap every page's frontmatter in <!-- ... --> so the renderer skips it while LLMs and grep can still parse the metadata from the source. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 16, 2026
  • Restructure wiki for LLM-friendliness: frontmatter, signatures, index Add machine-readable navigation aids and apply consistent structure across all pages so LLMs (and humans) can ground answers without inferring the schema each time: - New Glossary.md disambiguates overloaded terms that intentionally remain distinct (ar / ar_order / AR / "ar", ETS letter codes, state-component names overlapping parameter names). - New Roadmap.md collects R-only items and Python-port gaps so reference pages no longer carry TBA markers. - New llms.txt provides a flat index of every page with slug, category, status, R/Python symbols, and one-sentence purpose. - Every content page now opens with YAML frontmatter (title, slug, summary, status, applies_to, related) so retrieval can hit metadata instead of parsing the page. - [[WikiLink]] syntax replaced with relative-path links ([Page](Page.md)); R double-bracket subscripts preserved. - Inline "Read more in Svetunkov (2023), Section X" prose moved into a consolidated ## References block per page. - Function pages (ADAM, AutoADAM, ES, CES, MSARIMA, SSARIMA, GUM, SMA, OM, msdecompose) carry a ## Function signatures block with verbatim R and typed Python signatures near the top. - Parameter pages (Model-Specification, Orders-and-Lags, Loss-Functions, Bounds, Initialisation, Persistence, Explanatory-Variables, Model-Estimation) carry an ## At a glance strict table summarising argument name, type, default, valid values, and applies-to. - Home.md trimmed: long method tables that duplicated per-function pages replaced with compact "where to look" tables; prominent links to Glossary, Roadmap, and llms.txt added. - TBA cells removed from reference pages (GUM, SSARIMA, Scale-Model, Simulation-Functions, Orders-and-Lags, Explanatory-Variables, Model-Estimation, Model-Information, Model-Specification) and tracked on Roadmap.md instead. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 16, 2026
  • R-Python-differences: CES partial/full fixed via smoother="global" swap The CES partial/full divergence section now reports the resolution: both R and Python's CES now call msdecompose with smoother="global" (routed through the shared olsCore.h backend) instead of the historical default "lowess". R-parity CES failure count drops from 84 to 18 (the residual 18 are quarterly-data optimiser-floor effects, all at logLik diff ~2.7e-5). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 10, 2026
  • R-Python-differences: document OM fsum + FI pristine snapshot and CES lowess root cause Two new sections under "What used to be the one remaining numerical gap": 1. OM Bernoulli likelihood + FI Hessian (intermittent demand) — documents the two OM-side fixes just committed in the smooth repo (math.fsum on the Bernoulli sum, pristine matVt / profilesRecentTable snapshot before NLopt so the FI Hessian sees the same buffers R re-creates fresh). 2. CES "partial" / "full" seasonality (stats::lowess ULP gap) — diagnostic chain showing the CES partial/full divergence traces all the way back to greybox.lowess vs R's stats::lowess: at a degenerate B both kernel inputs match byte-for-byte except matVt's seasonal-component initial state rows, which differ by ~3.5e-14; that traces to msdecompose seasonal output (smoother="lowess") which traces to greybox.lowess differing from R's stats::lowess by ~3e-13. Verified by subprocess- delegating to R's lowess: partial_airpassengers then matches R's logLik bit-perfect. Fix path identified (port clowess.f faithfully into greybox); tracked as a CES follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 10, 2026
  • R-Python-differences: document msdecompose summation-order parity fix Adds a section describing the second msdecompose gap (separate from the shared olsCore.h work): three reductions inside msdecompose summed in a different IEEE order from R, producing ULP-level seasonal differences that the undamped multiplicative-trend recursion in ETS(M,M,M) amplified into different NLopt basins. The Python fix replaces np.mean/np.nanmean with _fsum_nanmean (Shewchuk exact, matches R's LDOUBLE mean) at the two centring sites, and _r_filter_mean (R's stats::filter cfilter backward order) at the inner seasonal-mean site, restoring byte-identical msdecompose output on taylor and lining up MMM logLik with R on every previously-divergent configuration. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 10, 2026
  • R-Python-differences: multicov(analytical) is now ~1e-10, not ~5% Re-measured the analytical multicov gap on ANN / AAN / ARIMA after the shared olsCore.h landed: worst case is ~1e-14, mostly at or below 1e-15. The "≤5%" line was tracking the old OLS-ULP cascade into optimiser drift, not the multicov formula itself. Update the top "What is identical" table and the parity-test tolerance table. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 6, 2026
  • R-Python-differences: shared olsCore.h closes the optimal-initials vcov gap Both languages' msdecompose() now route their OLS through the same Armadillo pivoted-QR + scale-invariant rank cutoff (olsCore.h). The "~2% on optimal/two-stage vcov" line in this page is no longer current — max |ΔSE/SE| at R's B is 1e-8 on ETS scenarios and 1e-4 on ARIMA, the FD-Hessian discretisation floor. The page now documents the new bound, what still moves at the optimiser floor (confint bounds, ~2% on flat ARIMA cost surface), and the new aligned point under "What had to be aligned". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed Jun 6, 2026
  • Document simulate-function R/Python differences Retitle "R / Python numerical equivalence" → "R vs Python differences" and add a "Simulation functions and RNG-induced differences" section covering: * Why ``set.seed(n)`` in R and ``seed=n`` in Python produce different draws (Mersenne Twister vs. PCG64). * The plug-in-numbers trick (callable randomizer) that proves the C++ ``adamCore::simulate`` kernel is bit-equivalent on both sides when fed the same errors. * Tolerances achieved by each ``sim_*`` r_parity test (atol 1e-9 to 1e-10). * Three xfailed cases in ``test_om_simulate_r_parity.py`` — OM-fit parity, Python multiplicative-seasonal NaN, upstream R-side ARMA-simulate bug — with one-paragraph explanations of each. * Practical guidance for reproducing simulations across languages. Also extend the parity-test tolerance table at the end with the new sim_* / simulate.* rows and add the new test files to the list. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed May 29, 2026
  • Document the relative-step Hessian rule Catches the wiki up to the C++ change shipped in ad35f5c0: * Coefficients-and-Parameters.md: new "Finite-difference step rule" subsection under vcov() explaining h_i = ε^(1/4) * max(|x_i|, 1) and why initial="optimal"/"two-stage" needed it. * Visualisation-and-Output.md: note under summary() flagging that the SE column for initial="optimal"/"two-stage" now agrees with R to within ~2%, with a back-link to the mechanics page. * R-Python-differences.md: rewrote as a concise factual reference — the per-parameter relative step closed the residual ADAM optimal/two-stage vcov gap, the only remaining ULP-level divergence is the LSQ solver inside msdecompose (LINPACK QR vs LAPACK SVD). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

    @config-i1 config-i1 committed May 27, 2026
  • Additional losses in OM/OMG

    @config-i1 config-i1 committed May 27, 2026