Epistemic Grounding of AI Agents for Proteomics Software Development

[magnuspalmblad]

Title

Epistemic Grounding of AI Agents for Proteomics Software Development

Abstract

Agentic AI coding tools are rapidly becoming practical for generating bioinformatics software, yet they often lack domain-specific safeguards needed for building scientifically sound proteomics workflows. This project will develop a community-governed epistemic grounding document for mass spectrometry-based proteomics: a machine-readable and human-reviewable set of hard constraints, conventions, and best-practice defaults that AI coding agents must follow when generating or modifying software. During the hackathon, participants will define an initial scope, curate examples of non-negotiable proteomics validity rules as hard constraints, encode softer rules as convention parameters, test the document across common agentic coding environments, and evaluate the extent to which generated workflows respect established community practice.

The project is particularly well suited to EuBIC because it requires expert input from several areas of the domain, including mass spectrometrists, experimental method developers and data analysis experts (and yes, statisticians). A shared grounding document will likely only be useful and adopted if it reflects community consensus, is openly maintained, and is tested against diverse real-world proteomics coding and data analysis tasks.

Project Plan

The project will begin by defining the scope of a proteomics GROUNDING.md by clarifying which tasks, file formats, assumptions, and software-development contexts it should cover, and perhaps some topics should remain out of scope. Participants will then collect representative examples of domain-critical rules, including peptide-spectrum match validity, false-discovery-rate estimation and handling, mzML/mzIdentML/mzTab/SDRF-Proteomics interoperability, controlled vocabulary use, search-parameter reporting, quantitative proteomics assumptions, and reproducibility expectations.

During the developer’s meeting, the group will translate these examples into a structured first draft with three layers: non-negotiable scientific constraints, recommended implementation conventions, and illustrative test cases for AI coding agents. Teams of two participants will test the draft using common agentic coding environments on realistic proteomics programming tasks, record failure modes, and refine the wording to improve interpretability by both humans and machines.
The hackathon project will deliver a minimal public prototype, example prompts, benchmark tasks, and a governance proposal for community review. Follow-up work will focus on expanding coverage, aligning with existing HUPO-PSI and EuBIC resources and activities, establishing a lightweight review process, and creating versioned releases. Longer term, the project could provide a reusable template for other computational biology communities seeking trustworthy AI-assisted software development.

We enthusiastically welcome participation from anyone who wishes to contribute, regardless of experience or skill level. Each participant will get the opportunity to describe how they see contributing to the project so that activities can be aligned accordingly. We particularly welcome the fresh perspectives and ideas of those with no prior experience with AI agents, context/prompt engineering or proteomics standards.

Technical Details

The project will be language-agnostic in principle, but the initial prototype will focus on Python, reflecting their widespread use in proteomics data analysis and bioinformatics tooling. The core output will be a plain-text or Markdown GROUNDING.md document, accompanied by machine-readable variants in YAML or JSON for automated validation. Unlike the diverse SKILL.md contexts that already exist, including for using pyOpenMS, the GROUNDING.md will not contain code templates, but rather define the scope of what is scientifically valid, given our best understanding of the domain. However, supporting scripts will be developed in Python to lint the grounding rules, run example prompts, compare generated code against expected behaviors, and visualize benchmarks. Where relevant, tests will use common scientific software infrastructure such as pytest, GitHub Actions, and containerized environments.

The project will rely on open standards, public documentation, and small, well-documented representative datasets rather than largescale reprocessing of raw data. Example inputs may include mzML, mzIdentML, mzTab-P, UniProt-styled FASTA, spectral library, and tabular quantification files, including from affinity proteomics platforms, from public repositories such as PRIDE. Dependencies should remain lightweight and optional, with emphasis on interoperability with existing HUPO-PSI formats and community workflow tools. Benchmark tasks, prompts, expected outputs, and rule examples will be stored in a public GitHub or Codeberg repository to enable transparent reuse, review, and extension.

Contact information

Magnus Palmblad
Center for Proteomics and Metabolomics
Leiden University Medical Center
n.m.palmblad@lumc.nl

Benjamin A Neely
Material Data Division
National Institute of Standards and Technology - Charleston
benjamin.neely@nist.gov

[magnuspalmblad]

I forgot to mention it, but please see this preprint for a bit more background to the project.

[magnuspalmblad]

There will also be a BioHackathon Europe project on "epistemically grounded" agent-driven generation and maintenance of a KG of the ELIXIR ecosystem. I will bring the lessons learned from this to the EuBIC-MS Hacakthon, if applicable to connecting distributed and diverse data and information standards, best practices, guidelines, and theoretical works. This may be a stretch goal for this hackathon project however.