From Spreadsheet to Schema: Standardizing Quantitative Proteomics in AnnData

[lmboat]

Title

From Spreadsheet to Schema: Standardizing Quantitative Proteomics in AnnData

Abstract

Processed proteomics data is often reduced to a matrix of numbers, while the decisions that produced it, including aggregation, normalization, imputation, and batch correction, are lost or recorded inconsistently. AnnData is an appealing container for proteomics because it keeps data, sample metadata, and feature annotations together, but it does not provide a standardized, machine-readable way to describe how each representation was generated.

The HUPO PSI-AI Quantification Working Group is developing two complementary community standards to address this gap: the Quantification Controlled Vocabulary (Quant CV), which defines terms for quantitative data transformations, and MIAPE-AnnData, which specifies the minimum information and provenance structure that proteomics AnnData objects should contain. Both are tool-agnostic and community-reviewed; the next step is practical implementation.

This hackathon will turn these standards into working software by building a LinkML schema and validator, converters from real search-engine outputs, and examples that capture transformations as they occur. The goal is an end-to-end, validated, provenance-carrying AnnData workflow that others can reuse and extend. The project is especially well suited to a community because it spans proteomics tools, ontologies, data standards, and the scverse ecosystem, with modular contributions for participants across these areas.

Project Plan

During the hackathon, we will focus on six connected tasks that together establish an end-to-end workflow from proteomics search-engine output to a validated, provenance-carrying AnnData object:

1. Build the MIAPE-AnnData schema and validator. Translate the reviewed object structure into a LinkML schema and generate validation models that can automatically check whether an AnnData object follows the specification. The schema will reference Quant CV terms rather than duplicate the vocabulary.
2. Map SDRF sample metadata into obs. Build on PRIDE's existing SDRF validation work to preserve standardized sample metadata and ontology identifiers in AnnData. Where identifiers are not already available, we will investigate reuse of existing ontology lookup tools and flag unresolved values for curation rather than guessing.
3. Build converters for at least two major proteomics tools, initially FragPipe and DIA-NN. Convert native quantitative output into a consistent AnnData representation containing quantitative data, sample metadata, feature annotations, and provenance information.
4. Demonstrate automatic recording of a quantitative transformation. Implement a reference transformation, such as log2 transformation or median normalization, that records what was done using a Quant CV term at the time the transformation is performed.
5. Test the workflow on real proteomics data. Validate converted objects and use validation failures to identify converter bugs, unclear mappings, or gaps in the specification to bring back to the PSI-AI working group.
6. Assemble an end-to-end walkthrough. Create a reproducible example taking a real dataset through conversion, metadata mapping, validation, transformation, provenance recording, and re-validation.

After the hackathon, we plan to circulate the schema, validator, converters, and walkthrough within the HUPO PSI-AI community for broader testing and adoption. Feedback from real-world implementation will be used to refine MIAPE-AnnData and expand support to additional proteomics tools and datasets, with the goal of a public v0.1 release. We also plan to document our progress in a BioHackrXiv manuscript.

Technical Details

Programming languages / frameworks:

• Python
• LinkML for schema definition and code generation; generated validation models will reference a pinned release of Quant CV rather than redefine the vocabulary
• Pydantic and JSON Schema for generated validation models
• AnnData as the target data structure, within the scverse ecosystem

Existing software and standards:

• MIAPE-AnnData and Quant CV
• Existing PRIDE SDRF validation and ontology tooling
• FragPipe and DIA-NN quantitative outputs
• Quantitative transformations will be recorded as validated TransformationStep entries in uns['quantification_provenance'], with transformation types referenced using Quant CV terms

Datasets:

• Small, licence-clear proteomics reference datasets with matching SDRF metadata will be selected before the meeting
• Where possible, datasets will include FragPipe and/or DIA-NN outputs and will be provided with a pinned software environment for reproducible development and testing

Contact information

Lisa Boatner
Chan Zuckerberg Initiative
Redwood City, CA, USA
lboatner@chanzuckerberg.com

Ahmad Lutfi
Technische Universität Berlin
Chair of Bioanalytics
Berlin, Germany
ahmad.lutfi@tu-berlin.de

Kevin Yang
Broad Institute
Cambridge, MA, USA
yake@broadinstitute.org

Karin Schork
Ruhr University Bochum
Bochum, Germany
karin.schork@rub.de

Tine Claeys
VIB - UGent Center for Medical Biotechnology
Ghent, Belgium
tineclae.claeys@ugent.be

[wolski]

Very interesting proposal. Bringing AnnData, SDRF, and the Quant CV together in working code is directly relevant to APB.

APB already converts DIA-NN, FragPipe and other quantitative outputs used by ProteoBench and reads the corresponding search or workflow parameters. It keeps protein, peptide, peptidoform, ion, and fragment data linked and writes AnnData, MuData, Parquet, or DuckDB.

I would be happy to use APB as a test implementation while your LinkML schema develops:

• map validated SDRF sample metadata into obs through APB's existing annotation support
• record transformations with Quant CV terms
• validate APB-generated AnnData against MIAPE-AnnData

Vendor mappings live in rules, so adapting a format usually does not require parser changes. SDRF support, MIAPE-AnnData validation and Quant CV recording will still require code. The schema and validator would remain independent of APB.

We have already linked to #16 from #14, Quality control of biological plasma samples with APB. Would you be open to adding a reciprocal reference to #14?