Close audit-writers blind-spot for text-templated writers + lock in growth_media id patterns

[realmarcin]

Summary

Two small follow-ups to the schema-gap audit landed in #84:

• F7 (high) — scripts/audit_writers.py was blind to scripts/clean_metals_inplace.py, which mutates 149 community YAMLs via regex/line-edit substitutions + path.write_text(string). The auditor's _WRITE_TEXT_OF_YAML regex only matched path.write_text(yaml.dump(...)), so a high-risk writer with no validation, no curation trail, and no yaml-roundtrip safety was invisible. Broaden the heuristic so it now surfaces. (Does not convert the script — that's a separate, larger change.)
• Fix Regenerate HTML for all 61 communities including new EcoFAB #5 (low) — GrowthMedia.culturemech_id and GrowthMediaComponent.media_ingredient_mech_id had no pattern: declared, while the sibling RelatedMedia.culturemech_id / RelatedIngredient.mediaingredientmech_id slots do. Live data already conforms to the standard ^CultureMech:\d{6}$ / ^MediaIngredientMech:\d{6}$ — just lock it in.

What changed

1. scripts/audit_writers.py — extended looks_like_yaml_writer

New branch in the detector:

A script that references the kb/communities path literal and round-trips the same path variable through .read_text() and .write_text(...) is a community-YAML writer (text-templated in-place mutation).

Calibration:

• Across all scripts/ + src/communitymech/ only one new script matches: clean_metals_inplace.py.
• src/communitymech/visualization/umap_generator.py and src/communitymech/render_community_pages.py reference kb/communities and call .write_text(...) but write HTML output to a different out_path — no same-variable round-trip, so they don't match.
• src/communitymech/literature.py and src/communitymech/utils/media_linker.py have same-variable read_text / write_text round-trips but operate on cache files (no kb/communities reference) — they don't match either.
• audit_writers.py itself round-trips path and references kb/communities in its regex source, but is already self-excluded via path.resolve() == Path(__file__).resolve().

After-state audit row for the now-surfaced writer:

scripts/clean_metals_inplace.py  yes  no  yes  no  no

(writes_yaml=yes, appends_curation_history=no, has_write_safeguard=yes — it does have --dry-run, validates_before_write=no, wired_into_just=no.)

2. src/communitymech/schema/communitymech.yaml — patterns added

GrowthMediaComponent:
  attributes:
    media_ingredient_mech_id:
      ...
      pattern: "^MediaIngredientMech:\\d{6}$"

GrowthMedia:
  attributes:
    culturemech_id:
      ...
      pattern: "^CultureMech:\\d{6}$"

Schema regenerated via just gen-python; the four compiled regexes are now in src/communitymech/datamodel/communitymech.py (two new lines for the GrowthMedia* slots, in addition to the two already present for RelatedMedia/RelatedIngredient).

3. reports/pipeline_writers_audit.tsv — refreshed

Now 16 rows (was 15). The added row is clean_metals_inplace.py.

Before/after

check	before	after
audit_writers TSV rows	15	16
clean_metals_inplace.py surfaced	no	yes
validate_strict ERROR rows	0 / 265	0 / 265
pytest tests/	136 pass / 9 skip	136 pass / 9 skip
ruff check scripts/audit_writers.py	clean	clean

Deferred

Converting scripts/clean_metals_inplace.py itself to a yaml.safe_load → mutate-dict → write_validated_community flow is not in this PR. That change is more invasive (149-file regex surgery to load/dump round-trip, with quoting / float-rendering / comment-preservation pitfalls). Now that the audit surfaces it, it's tracked in the standard backlog. From the audit's remaining un-gated writers, the next priority candidates are:

• intelligent_snippet_fixer.py (no safeguard, no validation, no curation)
• enhance_strain_data.py (no safeguard, no validation, no curation)
• add_evidence_source.py (safeguarded but no validation, no curation)
• clean_metals_inplace.py (this PR's new finding — --dry-run already in place)

Test plan

• uv run python scripts/audit_writers.py | grep clean_metals_inplace returns a row
• No new false-positives surfaced in the audit TSV (rows = 15 + 1 = 16; the new row is the target)
• uv run python scripts/validate_strict.py --quiet → 0 ERROR rows / 265 files
• uv run pytest tests/ -q --no-cov → 136 passed, 9 skipped
• uv run ruff check scripts/audit_writers.py → clean
• Regenerated datamodel/communitymech.py contains four ^CultureMech:\d{6}$ / ^MediaIngredientMech:\d{6}$ compiled patterns

[Copilot]

Pull request overview

This PR tightens audit coverage for community-YAML “writers” that mutate YAML via raw text round-trips (read_text → write_text) and strengthens schema validation by enforcing ID regex patterns for GrowthMedia-related cross-repo identifiers.

Changes:

• Expanded scripts/audit_writers.py writer detection to include in-place text-templated mutations targeting kb/communities (catches clean_metals_inplace.py).
• Added pattern: constraints for GrowthMedia.culturemech_id and GrowthMediaComponent.media_ingredient_mech_id in the LinkML schema and regenerated the Python datamodel.
• Refreshed reports/pipeline_writers_audit.tsv to include the newly detected writer.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
src/communitymech/schema/communitymech.yaml	Adds regex patterns to lock in GrowthMedia* ID formats consistent with existing related-* slots.
src/communitymech/datamodel/communitymech.py	Regenerated datamodel to include compiled regex patterns for the updated slots.
scripts/audit_writers.py	Broadens writer heuristics to detect community YAML mutations done via text round-tripping.
reports/pipeline_writers_audit.tsv	Updates the audit report to include scripts/clean_metals_inplace.py.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.