Layout fingerprints and a cached coordinate store for placing channels from irregular coords #1004
d-chambers
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Problem
Spool-level selection on inventory-defined coordinates (
spool.select(borehole=...)) works from the index, which stores only<dim>_min,<dim>_maxand<dim>_stepper patch. Channel membership is decided by rebuilding the sample grid arithmetically (_placed_rows,dascore/core/_spool_inventory.py). When a coordinate is not aCoordRangethere is no step, so there is no grid, and the operation refuses:Refusing is right — guessing a step would silently trim the wrong channels — but the consequence is that a format storing an explicit, non-uniform coordinate array cannot use these operations at all. Ran into this on Febus BOTDR files whose distance axis is a 0.1 m grid quantized to float32 (steps 0.099853–0.100098), so it types as
CoordMonotonicArray.Idea
scan_payloadsalready returns full coordinate managers, anddc.scanreduces them to envelopes and discards the arrays. The exact values are in hand at scan time and thrown away.scan_payloadsand populate. Place from the real values, so nothing is inferred and the objection above does not apply.Because the key is content-addressed, patches sharing an instrument configuration share one entry — 48 files collapsed to a single array in my case. It is a memo, not meaning: it cannot change an answer, only the time to get one. So it should not live on the spool, take part in equality, or be propagated across derivations, unlike
_inventory.The fingerprint column is the piece that cannot be retrofitted lazily — at query time you hold a row, not an array — so it is worth landing first even if the cache follows later.
Notes and open questions
hash(). A collision misplaces channels silently, which is the failure the current refusal exists to prevent._placed_rowsalready caches that tier by(id(context), axis, low, high, step).FiberIO.scan(snap=True)already lets a format present a stored axis as an idealized uniform range, but appears aimed at time. Whether near-regular distance axes should snap there is a per-format question.Against
dev.All reactions