Skip to content

physprobe v1.1 — course release

Latest
Latest

Choose a tag to compare

View all tags
@msannikov03 msannikov03 released this 23 Oct 01:10
· 2 commits to main since this release
v1.1
e2dab7d

physprobe v1.1 — course release (23 October 2026)

Question. Do frozen video representations (V-JEPA 2, DINOv2, random-init controls, the Cosmos video tokenizer) linearly encode the hidden mass and friction of a visually identical cube pushed in MuJoCo, and does a touch signal add to them?

Answer. Yes for vision: V-JEPA 2 reaches test R² 0.997 (ln m) / 0.977 (μ); controls show the probe reads kinematics (identical first frame → 0; fixed-speed slide → no mass; pooled features keep position). Raw pixels do as well (0.999 / 0.982), so a clean scene does not rank encoders (as CALIPER found). Friction needs the time profile; mass does not. Pretraining adds +0.08…+0.16 R²(μ) over random weights at matched pooling. The Cosmos tokenizer latent carries the physics less linearly (0.943 / 0.682 at a 4×4 grid). A realistic touch signal without absolute scale reaches 0.521 / 0.452 alone and adds +0.005 to V-JEPA 2, +0.11 to DINOv2. One leak (post-stop solver residual in the contact baseline) was caught by review and corrected; the write-up lists all corrections.

What is in this release

  • Code: simulator, feature extraction, baselines, probes, reports, tests (python -m pytest tests/ -q), scripts/reproduce.sh.
  • Results: results/{push,slide,poscheck,quicklook}/ (tables, gates, every fit as CSV, figures).
  • Docs: README.md, docs/RESULTS_2026-10-22.md, docs/proposal_v2.md, CONTRACT.md.
  • Demo: notebooks/demo_two_cubes.ipynb.
  • Data: features and per-clip signals on Hugging Face, businesslion/physprobe-features (tag v1.1, CC BY 4.0). With them, scripts/reproduce.sh --from probes reproduces every number on a CPU in about 15 minutes (scripts/compare_results.py checks the match).
  • Slides: attached below (HTML with speaker notes; PDF).

Licence: MIT (code). Authors: Mikhail Sannikov, Ilya Mikhalchuk.

Reproduce on a CPU (verified in a clean room from this tag: every number identical, 11 min after download):

git clone https://github.com/msannikov03/physprobe && cd physprobe
pip install -r requirements-probe.txt huggingface_hub==1.20.1
python scripts/download_features.py        # resumable; avoids the anonymous Hugging Face rate limit (HF_HUB_DISABLE_XET=1)
export SIM_PY=python FEAT_PY=python && source env.sh && scripts/reproduce.sh --from probes
python scripts/compare_results.py --rev v1.1
Assets 6
Loading