Integration of NIMAA-based missingness structure analysis as a new ProteoBench module for proteomics benchmarking

[jafarilab]

aim of the new module

This proposed module aims to introduce structured missingness analysis into ProteoBench using the NIMAA (Nominal Missing Data Analysis) framework.

What is its aim?
The module will quantify, visualize, and compare patterns of missing values in proteomics outputs across different software tools using a graph-theoretic representation of nominal data.
What is its added value compared to already available modules?
Existing ProteoBench modules primarily focus on identification rates, quantification accuracy, and error distributions. However, they do not explicitly characterize the structure of missingness, which is a key property of proteomics datasets and strongly influenced by software-specific processing, filtering, and scoring strategies.

This module adds a complementary dimension by:

Treating missing values as informative signals rather than noise
enabling comparison of structural bias between tools
providing network-based metrics (e.g., nestedness, modularity) that capture hidden structure in proteomics outputs

This creates a new benchmarking axis: data structure integrity, not only data completeness.

full description of the new module

1. What file would be available for download on the module website?

The module would provide:

processed ProteoBench benchmarking outputs (protein intensity matrices per tool and dataset)
standardized binary or quantitative presence/absence matrices (proteins × samples)
graph representations (bipartite edge lists derived from missingness patterns)
NIMAA-derived summary statistics files (CSV/TSV)

Data source: ProteoBench benchmark datasets (e.g., DDA/quantification outputs already integrated in ProteoBench).
If needed, outputs can be versioned and deposited in Zenodo via ProteoBench releases to ensure persistent identifiers.

Rationale: these datasets already exist within ProteoBench workflows, and the module operates as a post-processing analytical layer rather than a new dataset generator.

2. What would users need to do?

Users would:

Select a ProteoBench dataset (e.g., DDA quantification output)
Choose software outputs to compare (e.g., MaxQuant, Proline, Spectronaut, etc.)
Run the NIMAA module to:
convert matrices into bipartite graphs
compute missingness structure metrics
visualize structural differences between tools

No additional preprocessing beyond standard ProteoBench outputs would be required.

3. What would be the type/format of files used as input for the module?
Protein quantification matrices (CSV / TSV)
Format: rows = proteins or peptides or PSMs, columns = samples/conditions
Values: intensity values or NA (missing values)
Optional metadata file (organism, condition, replicate structure)
4. What metrics would be calculated and how?

Using NIMAA-based graph representations:

Relative missingness
proportion of NA entries per dataset
Nestedness (temperature score)
quantifies the hierarchical structure of missing values
Modularity
detects clustering of proteins/samples based on missingness patterns
Submatrix extraction (rectangular maximum)
identifies the largest complete block of quantifiable proteins
Permutation-based clustering tests
evaluates whether missingness or labels are randomly distributed or structured
Graph similarity metrics
compares structural similarity between software outputs
5. Proposed plot for metrics visualization

We propose the following visual outputs:

Heatmap of protein × sample matrices highlighting missing values
Bipartite graph representation of proteins and samples
Side-by-side comparison of:
nestedness scores
submatrix sizes
missingness proportion
“Bird’s-eye view” summary plot showing:
complete vs incomplete submatrix structure per tool
Optional network visualization of clustered proteins based on shared missingness patterns

potential external expert(s)

Lennart Mertens
Laurent Gatto

Will you be able to work on the implementation (coding) yourself, with additional help from the ProteoBench maintainers?

• yes
• no

any other information

This module is based on the recently published framework “Informative Missingness in Nominal Data: A Graph-Theoretic Approach to Revealing Hidden Structure” (Computational and Structural Biotechnology Journal, 2026, DOI: 10.34133/csbj.0099), developed in collaboration with Veit Schwämmle. The proposed NIMAA-based ProteoBench extension builds directly on this methodological foundation and adapts it to proteomics benchmarking applications.

[mlocardpaulet]

Hi, thanks a lot for this proposal. Missingness is indeed an interesting aspect of QCs.
Reading your proposal, I was wondering if it needed to be a new module. If I understand well, you can calculate the metrics once for all the data points that are submitted to the modules of interest, and generate a static report with figures (like the pmultiqc report)?
If this is the case, we could use the same srategy.
Currently, if you go to any multi-species module (for example the Quant LFQ DIA ion Astral, and go to the top-menu tab View Single Result, you can then select a point to plot the corresponding QCs (by default it is the workflow results that you loaded locally, if you loaded one, if not, you can select any public workflow).
It takes a bit of time to print the plots, and at the bottom of the page you can see the following:

It allows to generate the report (this takes a while) and download it. We intend to generate reports for all public points (#996), but this is work in progress.
Maybe we could apply a similar strategy to your plots?

It would be really nice to discuss this with you at one of our online meeting (that you can join any time anyway).

[jafarilab]

Hi,

Thanks for the suggestion. I completely agree with this strategy.

My idea was also to calculate the missingness metrics once from the submitted data and generate a static QC report with the corresponding figures, so reusing the existing report-generation framework makes a lot of sense.

Thanks for pointing me to the "View Single Result" functionality. It seems very well aligned with what I had in mind.

I'd be happy to discuss this further in one of the online meetings.