Experiments

Experiments -- the notebooks behind every number

Every number the experiment and domain pages quote comes from a notebook under packages/dtfit-experimental/experiments/. A notebook is one experiment and its report: the claim at the top with what would falsify it, the code that measures it, the numbers it prints, the reading of those numbers, and the limits that were not tested. A notebook is edited by hand and re-executed; nothing generates it, and the pages here are summaries that link to it.

Layout

packages/dtfit-experimental/
  experiments/
    README.md       # the index: claim, runtime and data needed, per notebook
    method/         # 11-19: what the image is and where each form of the method stops
    technology/     # 21-24: the filter, the reduce at scale, the stochastic image, the footprint
    realdata/       # 31-36: reference datasets, forecasting, GPS, the two archives, real stochastic series
    results/        # the tables the notebooks export, small CSV files, tracked
    data/           # downloaded datasets, not tracked
  src/dtfit_experimental/study/   # what the notebooks import: baselines, simulators, loaders, metrics
packages/dtfit-hardware/experiments/rig.ipynb   # the recorded drives and the on-chip filter

Running

pip install -e "packages/dtfit-experimental[bench]"
python -m dtfit_experimental.study.download_data     # fetch the datasets, once

jupyter lab packages/dtfit-experimental/experiments/method/12_discrete_image.ipynb
# or headless, outputs written in place
jupyter nbconvert --to notebook --execute --inplace \
    packages/dtfit-experimental/experiments/method/12_discrete_image.ipynb

DTFIT_QUICK=1 shrinks every notebook to a run of seconds; the package's tests/test_notebooks.py executes each committed notebook that way, and a check cell that no longer supports its claim fails the run. DTFIT_DATA moves the dataset directory. A section on real data names its source and skips with a visible message when the data is absent.

Index

The summaries: method studies for notebooks 11 to 19, the domain pages for the rest.

method/

Notebook	Establishes
11_source_forms	the three source criteria (spectrum balance, monomial integral, equal areas) against the image route, and where each stops
12_discrete_image	the image is the discrete statistic: zero bias and the pointwise fit's error on uniform, clustered and random grids, against the quadrature reading of the same criterion
13_two_bases	Legendre against block: reconstruction by coefficient count, Gram conditioning by order and grid, efficiency
14_robust_image	robustness is the reweighted statistic in either basis; the plain block fit has none; bursts
15_families	sixteen model families and the 25-scenario accuracy corpus against NLLS: recovery, noise sweep, forecast, sampling regimes, model mismatch
16_weak_form	weak-form ODE identification against finite differences and the solved fit; the seeded fit; where it breaks
17_evolution_matrix	every historical stage of the method on one problem set: batch, streaming and map-reduce
18a_block_basis_synthetic	where the block basis is ahead, needed, at parity and behind: known epochs, window alignment, aggregated data, gapped grids, stored numbers
18b_block_basis_real_data	the same map on 120 GPS station series and on hourly temperature read as 3 h and 6 h means
19_adaptations_in_trial	each adaptation of dtfit_experimental against the plain route on its own signal class, with a verdict

technology/

Notebook	Establishes	Summary
21_filter_form	the filter on the window image against a parameter-space EKF: accuracy by tuning, anomalies, dropouts, jump detection, cost	Embedded control
22_scale	exact merge, a lost shard, float32 numerics, memory against record length, fit_many, the GPU projection, the surrogate trap	Big data
23_stochastic_image	the second-order image: estimators by route, the regime router over 56 cases, the online filter, the generator	Stochastic series
24_embedded_footprint	state size, step cost and worst step of the filter against Kalman and refitting, by microcontroller	Embedded control

realdata/

Notebook	Establishes	Summary
31_reference_datasets	NIST StRD from both certified starts, Puromycin, two real growth rates	Parameter estimation
32_forecasting	twelve series against the forecasting baselines, the LTSF protocol, traffic	Forecasting
33_gps_simulation	a simulated maneuvering target from GPS fixes and a 9-DOF IMU against Kalman-CA and a coordinated-turn EKF	Real-time GPS
34_gps_benchmark	two literature trajectories and two GSDC trips with RTK truth, where the baselines win	Real-time GPS
35_archive_showcase	the NOAA and NGL archives reduced on a workstation and refitted on a Raspberry Pi 5	Archive showcase
36_stochastic_real	the stochastic router on seven real records against the random walk and the classical forecasters	Stochastic series

hardware

Notebook	Establishes	Summary
rig	three recorded drives against Kalman-CA and CT-EKF, and the on-chip float32 filter: cost, memory, agreement with float64	GPS hardware rig

Baselines

The established methods each experiment is scored against live in dtfit_experimental.study.baselines and dtfit_experimental.study.classical_stochastic; Baselines documents them. NLLS is the reference throughout: where theory gives the image no advantage the notebooks report parity as parity.