Skip to content

Harden MNIST trace-log diagnostics - #42

Draft
charlesmartin14 wants to merge 12 commits into
mainfrom
agent/harden-mnist-trace-log
Draft

Harden MNIST trace-log diagnostics#42
charlesmartin14 wants to merge 12 commits into
mainfrom
agent/harden-mnist-trace-log

Conversation

@charlesmartin14

Copy link
Copy Markdown
Member

Purpose

Harden the existing MNIST/MLP3 matrix trace-log diagnostic without changing its scientific definition or replacing WeightWatcher's detX calculation.

Changes

  • require the retrieved ESD to contain exactly min(weight.shape) eigenvalues for strict baseline measurements
  • reject non-finite, zero, or negative eigenvalues instead of silently filtering them and renormalizing a reduced positive-rank spectrum
  • require each WeightWatcher layer to map unambiguously to a two-dimensional model parameter
  • audit that WeightWatcher's rescaled eigenvalues sum to the full spectral dimension
  • reject out-of-range detX_num and num_pl_spikes values instead of clipping them into a valid range
  • preserve the existing midpoint rule and trace-log formula exactly
  • add analytic regression coverage for total trace-log, per-eigenvalue trace-log, and geometric mean
  • add failure tests for rank deficiency, incomplete ESDs, invalid boundaries, inconsistent ERG gaps, and incorrect normalization

Deliberately unchanged

  • WeightWatcher remains authoritative for detX_num, num_pl_spikes, and ERG_gap
  • no independent cumulative-log detX boundary is introduced
  • the midpoint retained rank remains floor((detX_num + num_pl_spikes) / 2)
  • optimizer mathematics, hyperparameters, notebook source, output notebooks, and committed experiment results are unchanged

Validation

  • git diff --check
  • full source and test compilation
  • 7 focused trace-log tests passed
  • complete baseline suite: 67 tests passed
  • pip check: no broken requirements

The final PR diff contains only:

  • baseline/rg_baselines/diagnostics.py
  • baseline/tests/test_diagnostics.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant