Home

APPN DataStorage Wiki

Welcome to the APPN Generic File Storage wiki. This repository provides a standardised data structure and automation scripts for managing research data storage across APPN (Australian Plant Phenomics Network) application nodes.

Overview

The project automates the creation and management of a hierarchical folder structure for phenomics research data. It handles folder generation, metadata files (YAML/JSON), field logs, and git integration for tracking changes.

Quick Start

1. Fork and clone the repository
2. Set up a conda environment:

   conda create -n datastorage python=3.12 numpy pandas pyyaml gitpython git
   conda activate datastorage

3. Edit NodeSummary.yaml to define your nodes and sensor platforms
4. Run python ProjectBuilder.py
5. Fill in the generated _ProjectsSummary.csv with sensor flags (TRUE/FALSE)
6. Re-run python ProjectBuilder.py to build out project folders

Wiki Pages

Page	Description
Folder-Structure	The hierarchical folder layout and naming conventions
ProjectBuilder	The main automation script for folder and metadata creation
NodeSummary-Configuration	How to configure nodes and sensor platforms
Code-Modules	Scripts in the Code/ directory (QA, extraction, utilities)
Core-Functions	Shared utility functions used across the project

Key Files

File	Purpose
ProjectBuilder.py	Main automation script
NodeSummary.yaml	Node and sensor platform definitions
FolderStructureInfo.txt	Reference for the folder naming convention
Code/	Processing scripts, QA tools, and shared functions

License

This project is released under the MIT License.