Extend code/data separation to config: make COUNTRIES_ROOT overridable (LSMS_COUNTRIES_ROOT) + declarative per-feature framework behavior

[ligon]

Summary

When the code/data separation was done (the data_dir / LSMS_DATA_DIR cache override), config was not given the same treatment. The country config tree is still bundled in the package and located install-relative, which couples "where the package is installed" to "which config tree gets read at runtime." This surfaced concretely during the 2026-06-14 parity-loop pass as the reason worktrees can't self-verify.

The smell

# lsms_library/paths.py
COUNTRIES_ROOT = Path(__file__).resolve().parent / "countries"

The country configs (countries/{C}/_/data_info.yml, data_scheme.yml, categorical_mapping.org, per-country .py) live inside the package and are found relative to the package's install location. Because the package is installed editable via a .pth pinned to the main checkout, Country('Uganda') always reads <main-checkout>/lsms_library/countries/Uganda/… regardless of CWD, PYTHONPATH, or which git worktree you are in.

Consequence: a worktree's config edits are invisible to a functional build (Country(C).feature() runs the main checkout's config), so worktree-based parallel development / verification is impossible without giving each worktree its own venv. The asymmetry is the tell — data was already made relocatable for exactly this reason; config was not.

Proposed change (high-leverage, low-risk)

Make COUNTRIES_ROOT overridable the same way data_dir already is — env var → config file → package-relative default:

COUNTRIES_ROOT = Path(
    os.environ.get("LSMS_COUNTRIES_ROOT")
    or _config.get("countries_dir")
    or Path(__file__).resolve().parent / "countries"
)

Benefits:

• A worktree sets LSMS_COUNTRIES_ROOT=<worktree>/lsms_library/countries and the live package reads that tree → worktree/parallel-branch dev + verification become first-class.
• Decouples install location from the config tree under development (e.g. pip-installed release pointed at a config checkout).
• Default unchanged → zero cost to the common case; override is opt-in (same trade-off data_dir already makes).

Audit for any other Path(__file__).parent-relative resource resolution while here.

Companion smell: hardcoded per-feature framework behavior

_no_v_join = {'sample', …, 'livestock', 'income'} and the _FOOD_DERIVED / _ROSTER_DERIVED dicts in country.py are hardcoded sets/dicts of feature names. A feature's v-join (and derived) behavior is a property of that feature and belongs in its data_scheme.yml (e.g. join_v: false), not in a framework set. During the parity loop a worker edited country.py to add crop_production to _no_v_join (a scope violation that had to be reverted) — exactly the failure mode this creates. Making per-feature behavior declarative means adding a feature never requires editing framework code, removing both the merge contention and the temptation to touch shared files.

General principle

Path(__file__).parent / "resource" is fine as a default, never as the only way to locate something you'll want to develop against, override, or ship separately. Config trees, feature registries, and reference data should resolve through config/env with the install-relative path as fallback. When code, declarative config, and data are separable at the resolution layer, parallel/worktree/CI workflows stop fighting the package.

Scope / sequencing

• Item 1 (LSMS_COUNTRIES_ROOT override) is the high-leverage, standalone change — small, consistent with the existing data_dir precedent, retires the worktree limitation.
• Item 2 (declarative per-feature v-join / derived behavior) is the natural follow-on that ends shared-framework-file edits during feature work.
• Implement after the current WB-parity loop pass completes (these touch paths.py / country.py, framework files the loop is building against).

Context + the by-gap architecture that worked around this: slurm_logs/2026-06-13_wb_incidence_map/PARITY_LOOP_DESIGN.md ("ARCHITECTURE USED" section).

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

[ligon]

Cross-link: this overlaps #279 ("bespoke label schemes without modifying the package tree") — same underlying need (configurable resource/config location so a checkout/worktree can carry its own labels without editing the installed package). The LSMS_COUNTRIES_ROOT override here is a concrete implementation path for #279.

[ligon]

Resolved: Item 1 (LSMS_COUNTRIES_ROOT config-tree override, #452) and Item 2 (declarative no-v-join in data_info.yml, #455) both merged. The _FOOD_DERIVED/_ROSTER_DERIVED declarative piece was intentionally left as code (each derived table requires a transform function; consistent with the 2026-04-18 !derived rejection). The worktree-based config-override dev flow was validated end-to-end this session. — triage 2026-06-15