Code Modules

Code Modules

The Code/ directory contains processing scripts, QA tools, and one-time utilities. Scripts are organised into prefixed subdirectories.

Directory Structure

Code/
├── DS02_DatasetQA/          # Dataset quality assurance scripts
│   ├── QA00_ELMvaliditation.py
│   └── README.md
├── DS03_PlotExtractionCode/  # Plot-level data extraction
│   └── PE00_LIDAR_extraction.py
├── functions/                # Shared utility functions
│   └── core_functions/
│       ├── parse_APPN_dataset_path.py
│       └── tests/
│           └── test_parse_APPN_dataset_path.py
└── OT00_OneTimeScripts/      # One-time migration/maintenance scripts
    └── OT00_RenameTiertoT.py

Naming Convention

Scripts use a prefix system:

Prefix	Meaning	Example
DS	Dataset processing step	DS02_DatasetQA
QA	Quality assurance	QA00_ELMvaliditation.py
PE	Plot extraction	PE00_LIDAR_extraction.py
OT	One-time script	OT00_RenameTiertoT.py

---

QA00 — ELM Validation Panel Extraction

File: Code/DS02_DatasetQA/QA00_ELMvaliditation.py
Author: Arden Burrell
Version: v1.0 (03.03.2026)

Automates the extraction and quality control of spectral data from ELM (Empirical Line Method) validation panels in hyperspectral imaging datasets.

What It Does

1. Searches for QC panel shapefiles matching *QC_*_Panel*.shp
2. Locates corresponding VNIR and SWIR orthomosaic rasters
3. Extracts spectral reflectance values from pixels within panel geometries
4. Saves extracted data as CSV or Parquet files
5. Generates visualisation plots showing spectral curves across panels and dates
6. Supports data sharing between nodes via save/load directories

Command-Line Arguments

Argument	Description
--path	Path to search for QA shapefiles (default: git root)
-f, --force	Force overwrite of existing output files
--type	Output format: csv or parquet
-s, --skipplot	Skip plot generation
--skip-processing	Skip raster processing; only load existing outputs
--save-dir	Save copies of extracted spectra for sharing
--load-dir	Load previously extracted spectra from another node
-v, --verbose	Detailed output

Usage

# Standard run
python QA00_ELMvaliditation.py

# Extract and save for sharing with other nodes
python QA00_ELMvaliditation.py --path /path/to/data --save-dir /shared/spectra

# Load external spectra and combine with local data
python QA00_ELMvaliditation.py --load-dir /shared/spectra

Dependencies

numpy, pandas, xarray, rioxarray, geopandas, shapely, matplotlib, seaborn, tqdm, GitPython

See Code/DS02_DatasetQA/README.md for tested package versions and conda environment setup.

---

PE00 — LIDAR Extraction

File: Code/DS03_PlotExtractionCode/PE00_LIDAR_extraction.py
Author: Arden Burrell & Richard Harwood
Version: v1.0 (09.03.2026)

Extracts plot-level data from LIDAR point clouds (GOBI and CALVIS sensors) using plot shapefiles.

What It Does

1. Crawls the dataset folder structure for LIDAR data directories
2. Validates plot shapefiles for each project site
3. Reads LAS point clouds and clips them to plot polygon geometries
4. Extracts DTM and DSM elevation values at each point location
5. Computes canopy height (Delta_z = z - DTM)
6. Saves results as CSV or Parquet tables

Command-Line Arguments

Argument	Description
--path	Root path to search (default: git root)
--no-git	Disable git operations
--save-dir	Save copies of extracted data
--load-dir	Load previously extracted data

Dependencies

numpy, pandas, xarray, rioxarray, laspy, geopandas, yaml, tqdm, GitPython

---

OT00 — Rename Tier to T

File: Code/OT00_OneTimeScripts/OT00_RenameTiertoT.py

A one-time migration script that renames folders starting with Tier to start with T (e.g. Tier0_raw → T0_raw).

Usage

# Use git root as search path
python OT00_RenameTiertoT.py

# Specify a custom path
python OT00_RenameTiertoT.py --path /path/to/data

The script searches recursively, lists all matching folders, asks for confirmation, then performs the rename.