Benchmarking feature finding algorithms for MALDI mass spectrometry imaging of tryptic peptides

[rodvrees]

aim of the new module

This module aims to benchmark feature finding algorithms for MALDI mass spectrometry imaging (MALDI-MSI) of tryptic peptides. Feature finding (the detection and delineation of peptide ion images from raw pixel spectra) is a foundational step in any MALDI-MSI proteomics workflow, yet no systematic cross-tool comparison exists. This module provides a ground truth dataset and standardized metrics to evaluate detection sensitivity, spatial fidelity, and quantitative accuracy of feature finding tools.

full description of the new module

1. What file would be available for download on the module website?

A timsTOF flex MALDI-MSI dataset of mouse liver tissue on which 8 synthetic tryptic peptides (sequence-unique vs the mouse liver proteome) have been spotted at a fixed concentration. The spotted peptides serve as ground truth features with known spatial location and identity. The dataset will be distributed in the community-standard imzML format.
VERY IMPORTANT NOTE: the dataset is preliminary. The final version will likely include a larger number of synthetic peptides and multiple concentration levels to enable quantitative accuracy benchmarking in addition to detection benchmarking. I probably will start design of this module with the "simpler" dataset to test things out.  Then the dataset will be replaced with the more "complex" dataset. Nothing will be made public until the final dataset is in place.

2. What would users need to do?

Download the imzML dataset from the module documentation.
Run their feature finding tool of choice (e.g., SCiLS, Cardinal, TIMSImaging, or a custom pipeline).
Export results as a tool-specific output or long-format CSV with columns: feature_id, mz, x, y, intensity, and optionally sequence and adduct.
Upload the CSV to ProteoBench together with a parameter/metadata file describing the tool settings used.

3. What would be the type/format of files used as input for the module?

Result file: long-format CSV (feature_id | mz | x | y | intensity), one row per detected feature per pixel.
Parameter file: tool-specific config or a structured metadata form describing, at minimum, the m/z tolerance used, peak picking algorithm, and spatial smoothing settings.

4. What metrics would be calculated and how?

Ground truth matching is performed by comparing detected feature m/z values against the theoretical [M+H]⁺ m/z of each synthetic peptide within a configurable tolerance. For each matched ground truth peptide, the following metrics are computed:

Detection rate: fraction of the 8 synthetic peptides\' features detected (sensitivity).
False discovery in background region: fraction of detected features in tissue pixels outside the spotted region that match a synthetic peptide m/z (specificity proxy).
m/z accuracy: median absolute m/z error of matched features vs theoretical values.
Ion image spatial fidelity: cosine similarity between the binarized detected ion image and the expected spot mask (derived from the known spotting coordinates). This is a metric specific to imaging and has no equivalent in existing ProteoBench modules.
Within-spot intensity CV: coefficient of variation of pixel intensities within the spotted region for each matched peptide (precision).

When concentration gradients are added to the dataset, a log2 ratio accuracy metric (epsilon, analogous to existing HYE modules) will be added.

5. Propose a plot for metrics visualisation

Main metric plot: scatter plot of ion image spatial fidelity (x-axis) vs detection rate (y-axis), one point per submitted tool/workflow. This provides an immediate overview of sensitivity vs spatial accuracy.
In-depth plots:

Ion image overlay: ground truth spot mask vs detected ion image for each synthetic peptide, per submission.
m/z error distribution: violin plot of per-feature m/z errors across the 8 peptides.
Within-spot CV: bar plot per peptide per tool.

potential external expert(s)

TBD. Developers of open-source MALDI-MSI feature finding tools (e.g., Cardinal, pyimzML ecosystem) would be the most relevant reviewers.

Will you be able to work on the implementation (coding) yourself, with additional help from the ProteoBench maintainers?

• yes
• no

any other information

The timsTOF flex in MALDI mode optionally produces trapped ion mobility (TIMS) data. Whether feature finders are expected to use ion mobility as an additional dimension (making this a 4D feature finding benchmark: x, y, m/z, 1/K₀) is to be decided based on tool support in the community.