Add permutation-column VIP / CV-beta null for the observational relate#468
Merged
Conversation
In the p >> n observational relate, even a descriptor with no real relationship earns a non-trivial VIP, so a magnitude threshold cannot come from the fit alone. `permutation_column_null` builds one from the data itself: it appends permuted "knockoff" columns - each a row-shuffled copy of a real descriptor, preserving the column's own marginal rather than simulating a distribution - refits the PLS relate over many permutations, and reports, per surviving descriptor, whether its VIP and cross-validated beta clear a high quantile of the knockoff null band. The function is decoupled from the influence gate. It takes an `ignore` list of descriptor names (by name, validated so a typo raises) that are dropped from the fit entirely - not merely skipped when building knockoffs - so a caller passes the gate-demoted spikes and calibrates only the survivors, undistorted. The knockoff count scales with the block (fraction + min floor for narrow data, optional cap) and `n_iter` sets the permutation count. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018M9u5EpSjHa3mjNaYDJ7qq
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Fit the augmented block directly and catch both LinAlgError and NotEnoughVarianceError, so a no-variance (constant) descriptor block degrades to an ok=False result instead of raising an uncaught error. Add tests for the degenerate block, the max_knockoffs cap, the too-few-descriptors return, and the min_knockoffs guard, covering the branches codecov flagged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018M9u5EpSjHa3mjNaYDJ7qq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
p >> nobservational relate, even a descriptor with no real relationship earns a non-trivial VIP, so a driver-magnitude threshold cannot come from the fit alone. Newprocess_improve.sensory.permutation_column_nullbuilds one from the data itself.n_iterpermutations, and reports, per surviving descriptor, whether its VIP and cross-validated beta clear a high quantile of the knockoff null band (vip_exceeds_null/cv_beta_exceeds_null).ignorelist of descriptor names (validated - an unknown name raises, so a typo fails loudly) that are dropped from the fit entirely, not merely skipped when building knockoffs, so a caller passes the gate-demoted spikes and calibrates only the survivors, undistorted. The knockoff count scales with the block (fraction, amin_knockoffsfloor for narrow data, optional cap);n_iteris the permutation count.random_state.Test plan
uv run pytest tests/test_sensory.py --no-cov(53 passed). New tests: genuine drivers clear the null while noise columns fall below them;ignoredrops named columns from the fit and output; an unknownignorename raises; identical output for a fixed seed; out-of-rangefraction/quantileare rejected.ignore, the surviving well-supported chemicals are calibrated against their own permuted-column null.uv run ruff check .anduv run mypy src/process_improvepass.Checklist
pyproject.toml(MINOR:1.57.0->1.58.0, new public API;CITATION.cffsynced)ruff check .passesCHANGELOG.mdupdated🤖 Generated with Claude Code
https://claude.ai/code/session_018M9u5EpSjHa3mjNaYDJ7qq
Generated by Claude Code