Skip to content

v0.2.0: lkj and residuals now sample what they claim

Choose a tag to compare

@microprediction microprediction released this 22 Sep 21:54
· 11 commits to main since this release

Two generators were not sampling the distribution their documentation described. Both are fixed, and the output for a given seed differs from 0.1.0 for these two ensembles only.

  • lkj now samples LKJ(eta). The Cholesky-row Beta draw was symmetric, Beta(b, b) with b = eta + (n-i-1)/2; LKJ (2009) requires Beta(i/2, eta + (n-1-i)/2). The old law was non-exchangeable and eta had no effect on it. The fixed lkj agrees with onion and vine to three decimals in E[det C] at every eta tested, and is exchangeable.
  • residuals has an observation-noise term. y = X @ coefficients was fit exactly by the regression, so the "residuals" were floating-point rounding error (rms ~1e-15 against a signal of order 1) and the output was chaotic at the ulp level. residuals_corr now takes noise=0.1, the noise standard deviation as a fraction of the signal's. The ensemble is intrinsically rank m+1 with m = int(sqrt(n+10)); the docstring says so.
  • nearest_corr docstring no longer claims Higham's nearest-correlation algorithm. It is a single spectral clip and rescale. No numerical change.
  • New tests in tests/corrgens/test_elliptope_measure.py check the measure rather than the matrix: E[det C] against LKJ theory, eta monotonicity, exchangeability. The pre-fix lkj fails all three while passing every shape, symmetry and PSD test.
  • The paper's n=30 tables, audit matrix and leaderboard in papers/measure_audit/ are regenerated against the fixed generators. The scale grid is being regenerated.