Design review: reference semantics, construction API, and implementation strategy

[d-chambers]

This summarizes a design review of the inventory proposal against DASCore's existing architecture (Patch/Spool immutability, coordinate management, directory indexing). Straightforward documentation fixes are in #2; this discussion covers the substantive design recommendations and the open questions that came out of the review. The mock API at the end shows the intended end-to-end user experience.

Overall assessment

The core model is sound: the StationXML-parallel layering, the "inventory stays external, patches carry only a data_source_id" rule, and the four independent interval tracks on OpticalPath all held up against the three worked examples (simple trench, NORFOX, tunnel). The recommendations below are about reference semantics, construction/modification ergonomics, and implementation strategy — not the model's shape.

R1: References are IDs; objects are accepted at construction and auto-registered

Shared assets (interrogators, cables, enclosures) must not be embedded inline in the tree. With immutable models, inline copies go stale: updating a cable in resources would leave old copies inside every FiberSegment that embedded it, and value-based replace() can't disambiguate identical copies.

Proposed semantics:

• Fields like container and interrogator are typed Cable | str (etc.) for construction convenience, but validation normalizes to a resource ID and auto-registers the object into Inventory.resources.
• The serialized tree stores only IDs; there is exactly one authoritative copy of each shared asset.
• Generated IDs are deterministic (name-derived slugs, not random UUIDs) so version-controlled inventory.yaml files diff cleanly.
• Lookups go through the inventory (inventory.resolve(id)); patch methods resolve internally so most users never touch this.

The trade-off: a detached Acquisition can't answer acq.interrogator.model without the inventory in hand. This mirrors ObsPy's response-attachment model and is the price of one-edit updates. This matters more now that concurrent routes are separate FiberArrays sharing instruments and cables (e.g., NORFOX's five arms referencing the same three interrogators).

R2: Modification is addressed verbs, not tree surgery

Hand-rebuilding nested tuples (fiber_array.optical_paths[:-1] + (closed, new)) should never be the documented workflow. Three layers:

1. Timeline verbs for field events — register_fiber_break(...), add_path_revision(...), end_acquisition(...) — each encapsulating "close the current revision, open the new one."
2. Addressed update — inventory.update(address, **changes) where the address is a data_source_id prefix or a resource ID.
3. Low-level escape hatch — .new() and a strict replace(old, new) that raises unless the match is unique.

R3 (open question): columnar tracks instead of per-interval objects

Geometry, coupling, and annotation tracks are currently tuples of small objects positioned partly by implicit order, where one wrong optical_length silently shifts everything downstream. But these objects are homogeneous table rows — and the field notes they come from are literally tables. The proposal: each of these three tracks becomes one table-like object per track with explicit distance columns (Geometry.from_table(df), Coupling.from_table(df), ...), while optical_components stays a tuple of typed objects (heterogeneous types, rich per-type fields, resource links).

Benefits: the authoring format, YAML storage format, and compute format become the same columnar structure; explicit distances fix the silent-shift failure; validate() checks per-track tiling rules (components/geometry/coupling tile [0, L], annotations may overlap).

Cost: a table row can't carry a rare rich one-off field the way an object could — overflow goes to an annotation or an extra column. This changes the data model, not just the API, so it needs a decision before spec changes.

R4: YAML/dicts are the primary authoring surface

Every nested field should accept a plain dict (native pydantic coercion), making Python construction just the YAML structure written inline — no imports of ~12 leaf classes. The documented workflow becomes: write inventory.yaml (plus CSVs that are the field notes), load, validate. Typed constructors remain for validation and autocomplete but move to an "advanced" section. Under this, the simple example collapses from ~120 lines of constructors to a ~40-line YAML file and dc.Inventory.from_yaml(...).

R5: Patch/spool surface stays small, explicit, and lazy

• A spool-attached inventory works lazily: it filters at the index level and resolves data_source_ids; it never eagerly enriches patches at instantiation (that would put geometry interpolation in every patch-load loop).
• One explicit patch enrichment method rather than three near-duplicates.
• Spool-level coordinate selection means intersection filtering only (keep patches whose resolved geometry intersects the query); channel trimming — including the non-contiguous unions a wandering path produces — is a patch-level operation. No unselect verb.
• Unmatched patches warn by default rather than being silently dropped (on_unmatched="drop" opt-in).
• Patches spanning a path-revision boundary need a documented policy: resolve at patch start time, warn by default, on_conflict="raise" available.

R6: Hard deployment physics fits existing primitives

Checking the tunnel example against the model, no new object types are needed — but two semantics need defining:

• geometry_type projection rules: linear → interpolate; point/coiled (e.g., a buried clump-sensor loop) → all channels in the interval get the constant coordinate.
• Helically wound cable: the optical-vs-axial distance factor is already representable in the geometry track's distance/coordinate pairs; add one informational field on FiberSegment (e.g. fiber_distance_factor) for strain/gauge-length interpretation. The geometry track remains the sole authority for coordinates.

The governing principle: every new physical situation should be answered by "which track does this go on," not "what new class do we need."

R7: Implementation strategy — immutable tree + compiled tables

• Inventory classes subclass DASCore's DascoreBaseModel (frozen pydantic, .new(), equality, serialization for free).
• Because objects are immutable, compiled artifacts can be cached per object: a source-id table (one dataframe resolving data_source_id + time → acquisition/path, enabling whole-index joins for spools) and per-path compiled tracks (sorted numpy (start, stop, payload) arrays; projection is np.searchsorted + np.interp, O(N log M) per patch).
• Updates rebuild only the spine from the changed node to the root; unchanged OpticalPath objects are the same Python objects across inventory versions, so per-path caches survive revisions.
• The .patch_attrs.yml directory sidecar needs its mtime treated like a data-file mtime for index invalidation, plus documented precedence over file-native attrs.
• Inventories are KB–MB scale; nothing here needs interval trees or a database.

Mock API

End-to-end user story under the recommendations above.

1. Author the inventory as YAML + field-note CSVs

# inventory.yaml
coordinate_reference_system: EPSG:4979
resources:
  int_das1000: {type: interrogator, manufacturer: Example Instruments, model: DAS-1000}
  cable_solifos: {type: cable, manufacturer: Solifos, model: BRUsens DAS AC3, fiber_count: 4}
networks:
  - code: XX
    fiber_arrays:
      - code: SIMP
        acquisitions:
          - code: RAW
            data_type: strain
            sample_rate: 250
            gauge_length: 2.0
            spatial_interval: 1.0
            interrogator: int_das1000
        optical_paths:
          - start_time: 2026-06-01
            optical_components:
              - {type: fiber, optical_length: 50, name: telemetry lead-in}
              - {type: connector, connector_type: E2000}
              - {type: connector, connector_type: E2000}
              - {type: fiber, optical_length: 720, cable: cable_solifos}
            geometry: {table: tap_points.csv}     # distance,latitude,longitude,elevation
            coupling: {table: coupling.csv}       # distance,optical_length,coupling_type,medium
            annotations:
              - {distance: 50, optical_length: 420, label: trenched}
              - {distance: 470, optical_length: 300, label: loose}

2. Load, validate, use with a spool

import dascore as dc

inventory = dc.Inventory.from_yaml("inventory.yaml")   # validates on load

# Lazy attachment: index-level filtering only, no per-patch work.
spool = dc.spool("/data/archive", inventory=inventory)  # warns on unmatched files

sub = spool.select(time=(t1, t2), latitude=(48.80, 48.81))  # intersection filter
patch = sub[0]  # instantiation adds nothing but data_source_id

3. Enrich a patch explicitly (the only hot-path cost, paid on request)

patch = patch.add_inventory_info(
    inventory,                      # optional if spool-attached
    distance="channel",             # map channel index -> optical distance
    coords=("latitude", "longitude", "elevation", "label"),
    attrs=("gauge_length", "interrogator.model"),
    on_conflict="warn",             # patch spans a path-revision boundary
)

4. Record field events with timeline verbs

# A break discovered in continuous data: closes the current path revision,
# opens a shortened one. One line, no tuple surgery.
inv2 = inventory.register_fiber_break("XX.SIMP", time="2026-06-12T10:30:00", distance=612.0)

# Correct a typo'd acquisition value, addressed by data_source_id.
inv3 = inv2.update("XX.SIMP..RAW", sample_rate=500.0, notes="corrected from field log")

# Update a shared resource once; every referencing array sees it.
inv4 = inv3.update("cable_solifos", owner="New Operator")

inv4.to_yaml("inventory.yaml")   # deterministic IDs + stable ordering -> clean git diff

5. Typed construction remains for programmatic/advanced use

from dascore.core import inventory as inv

path = inv.OpticalPath(
    optical_components=(lead_fiber, connector, sensing_fiber),
    geometry=inv.Geometry.from_table(tap_points_df),
    coupling=inv.Coupling.from_table(coupling_df),
).validate()

Open questions

1. Columnar tracks (R3) — accept the model change, or keep per-interval objects?
2. Deterministic ID scheme — name-derived slug rules and collision handling.
3. Exact signature of the single patch enrichment method (name, and whether distance mapping is folded in or stays separate).
4. Sidecar (.patch_attrs.yml) precedence and recursion rules.
5. Whether data_source_id code parts should carry FDSN-style length/charset restrictions for tooling compatibility.

---

This review was drafted with Claude Code working against the proposal site and the DASCore codebase.