Changelog

Folder Hierarchy Changelog

This page records changes to the APPN folder hierarchy only — that is, the structure described in Folder-Structure and the diagram on the Home page. Code-only changes, refactors, documentation tweaks, and CLI flag additions that do not alter what gets created on disk are intentionally excluded.

Each entry should describe:

• What changed in the on-disk folder layout
• Which sensors / levels are affected
• Why the change was made (one line is fine)
• A link to the relevant commit / PR if available

Entries are listed newest-first.

---

2026-08-13 — Run-level PlotExtracts/, SpectralIndices/ and GCP-QC artefacts

• New DS03 plot-extraction outputs: PE00_LIDAR_extraction.py (v2.0) and the new PE01_HyperspecPlotExtraction.py write per-run tables, provenance YAML sidecars, markdown reports and figures to T1_proc/PlotExtracts/ (Tier 1; T2_traits/ stays reserved for ML-model-derived products).
• New DS05 SI00_SpectralIndices.py writes spyndex index maps, JSON manifests, markdown overviews and figures to T1_proc/SpectralIndices/.
• The GCP QC pair (QA01_PointDistanceComparison.py per-run, new QA03_GCPRunComparison.py cross-run) adds stable-named QC_GCP[_{Product}]_distances tables + report JSONs inside T1_proc/QC_data/ and QC_GCP_* comparison tables/figures/report in the routed QCReports/ folders.
• Why: DS03/DS05/QA03 tooling ported from APPN-42-datastorage so plot extraction, spectral indices, and GCP accuracy tracking run at every node.

Affected paths:

.../run_XX/T1_proc/PlotExtracts/PE_LIDAR_points[…].parquet (+ _metadata.yaml)
.../run_XX/T1_proc/PlotExtracts/PE_{REGION}_{pixels|plot_metrics}[…].parquet
.../run_XX/T1_proc/PlotExtracts/PE_extraction_report[…].md + PE_figures/
.../run_XX/T1_proc/SpectralIndices/SI_{region}_{method}[…].{nc|tif} (+ _report.json, _overview.md, SI_figures/)
.../run_XX/T1_proc/QC_data/QC_GCP[_{Product}]_distances[_{extra}].{csv|parquet} (+ _report.json)
.../{Node}/Documents/QCReports/QC_GCP_*   and   .../{Project}/Documentation/QCReports/QC_GCP_*

---

2026-08-13 — Spectral-QC output folders (QC_Spectral_Tables/, QC_plots/, QCReports/)

• QA00_SpectralValidation.py (renamed from QA00_ELMvaliditation.py) now writes per-run outputs inside each run's T1_proc/QC_data/: extracted panel spectra in QC_Spectral_Tables/, a QC_spectra_report.json, and per-target figures in QC_plots/.
• The new QA02_SpectralRunComparison.py saves multi-run comparison figures to Documents/QCReports/ (node level) or Documentation/QCReports/ (project level).
• Why: DS02 spectral-QC overhaul ported from APPN-42-datastorage (per-run extraction/report split from multi-run comparison).

Affected paths:

.../run_XX/T1_proc/QC_data/QC_Spectral_Tables/QC_{ELM|VAL}[_{id}]_spectra_{VNIR|SWIR}[_gproN].{parquet|csv}
.../run_XX/T1_proc/QC_data/QC_spectra_report.json
.../run_XX/T1_proc/QC_data/QC_plots/<target>_spectra.png
.../{Node}/Documents/QCReports/
.../{Project}/Documentation/QCReports/

---

2026-05-20 — Site-level Documentation/Plot_Layout/ and Documentation/Trial_Info/

• ProjectBuilder.py now auto-creates two protocol-defined subfolders inside every site's Documentation/: Plot_Layout/ and Trial_Info/.
• A README.md template stub is seeded once in each subfolder (never overwritten) following the example in the Plot Delineation protocol.
• Plot polygon files ({YYYYSiteName}_plots.geojson + variants/sampling/gcp/sidecar) and the trial-info spreadsheet ({YYYYSiteName}_trial_info.csv) are authored by the operator. The script does not create or validate them — that check is handled by separate tooling.

Affected paths:

.../{Site}/Documentation/Plot_Layout/README.md
.../{Site}/Documentation/Trial_Info/README.md

---

2026-04-22 — Add RunOverview.csv at the date level

• A new RunOverview.csv file is now created inside each YYYYMMDD/ date folder, alongside FieldNotes.txt and the run_XX/ subfolders.
• One row per run, seeded with a Run index column and a RunFailed boolean column. Operators can add further columns (e.g. APEx experimental info) and they are preserved on subsequent script runs.
• Creation is controlled per field-day by the MakeTableFile column in FieldLog.csv (defaults to True when blank/NaN; set False to skip).

Affected path:

.../<sensor>/YYYYMMDD/RunOverview.csv

---

2026-04-22 — Add T0_raw/Vault/ for GOBI and CALVIS

• A new Vault/ folder is now created inside T0_raw/ whenever the sensor is GOBI or CALVIS.
• Files placed inside Vault/ are reserved as protected: they will be excluded from any future programmatic file cleanup or deletion routines.
• Mirrors the existing T1_proc/QC_data/ carve-out for the same sensor set.
• .graw folders should now be placed in T0_raw not T1_proc

Affected path:

.../<sensor>/YYYYMMDD/run_XX/T0_raw/Vault/   # GOBI, CALVIS only

---

Template for new entries

## YYYY-MM-DD — Short title

- What changed on disk.
- Sensors / levels affected.
- Rationale (1 line).

**Affected path:**

` ` `
.../path/that/changed/
` ` `