Folder Structure

Folder Structure

The APPN data storage follows a strict hierarchical folder layout. Each level encodes metadata in its name, making it possible to programmatically parse paths and extract project, site, sensor, and temporal information.

Hierarchy

node/
├── {NodeName}_ProjectsSummary.csv
└── YYYY_ProjectDesc{_I|E}{_Researcher}{_org}/
    ├── ProjectSummary.yaml
    ├── FieldLog.csv
    ├── YYYYSiteName{_F|C}/
    │   ├── Documentation/
    │   ├── Code/
    │   └── SensorPlatform/
    │       └── YYYYMMDD/
    │           ├── FieldNotes.txt
    │           ├── RunOverview.csv
    │           └── run_XX/
    │               ├── T0_raw/
    │               ├── T1_proc/        # + QC_data/ for GOBI, CALVIS
    │               └── T2_traits/
    ├── Documentation/
    └── Code/

Naming Conventions

Separator

Use underscores (_) to separate blocks of information within a folder name.

Node

The top-level folder identifying the APPN node. The university abbreviation comes first.

Example	Meaning
USYD_Narrabri	University of Sydney, Narrabri site

Project

Format: YYYY_ProjectDesc{_I\|E}{_Researcher}{_org}

Component	Required	Description	Example
YYYY	Yes	Year the project started	2025
ProjectDesc	Yes	Short description, no spaces	SIFcalibration
I or E	No	Internal or External	E
Researcher	No	Lead researcher last name (Title Case)	Salter
org	No	Organisation code (Alphanumeric string)	ABC123

Optional arguments must be programmatically identifiable when present.

Examples: 2025_Chickpea, 2025_TestProject, 2026_OZGrain

Site

Format: YYYYSiteName{_F|C}

Component	Required	Description	Example
YYYY	Yes	4-digit year	2025
SiteName	Yes	Name or code of the site	IAWatson
_F	No	Field site	2025IAWatson_F
_C	No	Controlled environment	2025IAWatson_C

Sensor Platform

The short name of the sensor platform. For field measurements (e.g. biomass cuts), use GroundTruth.

Currently defined platforms:

Sensor	Description
GOBI	GOBI hyperspectral
HIRES	High-resolution camera
M3M	M3M sensor
CALVIS	CALVIS hyperspectral
PHENOMATE	PhenoMate platform
MOLE	MOLE sensor
TEMS	TEMS sensor
PTEMS	PTEMS sensor
LITERAL	LITERAL sensor
H30T	H30T sensor
RHIZO	Rhizotron
FIELDOBS	Field observations

Date Folder

Format: YYYYMMDD — the date of field work or data collection, with leading zeros.

Notes:

• For continuous sensors (weather stations, soil moisture probes), this can be the first/last day of the month, the date logging started, or the date of data retrieval.

Run Folder

Format: run_XX — zero-indexed with leading zeros (run_00, run_01, etc.).

A "run" is a block of data processed together:

• For drones: an individual flight
• For soil radar: one pass with the tractor
• For other instruments: analogous acquisition blocks

Tier Folders (Data Processing Levels)

Each run contains three tier folders:

Folder	Purpose
T0_raw	Raw, unprocessed data as collected
T1_proc	Processed data (e.g. orthorectified, calibrated)
T2_traits	Derived traits and analysis outputs

Some sensors (e.g. GOBI, CALVIS) also have a T1_proc/QC_data/ subfolder for quality control data.

Optional Folders

At the project and site levels, optional Documentation/ and Code/ folders can store level-specific documents and scripts. The site-level Documentation/ and Code/ folders are created automatically by ProjectBuilder.py whenever a site folder is created.

Per-Day Files

At the date-folder level, ProjectBuilder.py can create:

• FieldNotes.txt — empty notes file for the day's work (controlled by the MakeNotesFile column in FieldLog.csv)
• RunOverview.csv — table with one row per run and a RunFailed boolean column (controlled by MakeTableFile)

Both default to being created unless explicitly set to False in the field log.

Path Parsing

The parse_APPN_dataset_path() function in Code/functions/core_functions/ can automatically extract metadata from any path in this structure. See Core-Functions for details.