Would a small release-audit report help with KG provenance and source metadata?

[ehsanestaji]

Hi BioCypher maintainers,

I am validating a small release-audit idea for biomedical knowledge graphs before building anything public.

I noticed two related BioCypher issues:

• Propagate provenances biocypher#56 on propagating provenance for nodes and relationships
• Strict mode biocypher#71 on strict-mode requirements such as source, license, version, and identifier

The idea is not to replace BioCypher or add another KG framework. I am trying to understand whether a compact release report would be useful around the edge of existing tools.

The report would be read-only over graph outputs plus a small release manifest. It could summarize things like:

• missing node or edge provenance
• missing source versions
• missing or unclear source licenses
• identifier and prefix consistency
• selected Biolink/KGX validation findings
• evidence metadata completeness
• policy tags for controlled or sensitive cohort/sample metadata

The output would be stable JSON and Markdown, with finding IDs that could be used in CI or release notes.

Example finding shape:

{
  "id": "BKG-PROV-001",
  "severity": "error",
  "message": "Edge is missing primary_knowledge_source.",
  "affected_element": "edge:e2",
  "suggested_action": "Add a primary upstream information resource using an InfoRes CURIE."
}

My questions:

1. Would this reduce real release or CI friction for BioCypher users?
2. Are these checks better handled inside BioCypher strict mode instead?
3. Which one or two checks would be most useful to validate first?
4. Which checks would be noisy, harmful, or too project-specific?

If this belongs upstream in BioCypher, that would be the preferred direction. I am trying to avoid creating a standalone tool unless there is clear pull from maintainers/users.

[slobentanzer]

Hi @ehsanestaji,
thanks for getting in touch, and I agree with your sentiment. I think it need not be exclusive; many things we developed downstream ended up moving up into the framework at some point where we saw the usefulness.

Such a report would definitely be useful, but I think much of the field has still not even arrived at the point of using strict mode. Not too early to be ready for it, though. As such:

1. Probably not, because of the low rate of strict mode use (due to lack of maturity in data pipelines).
2. A separate reporting tool can be more agile than implementing it in the core framework (which is large enough as it is). Another alternative could be the CLI we are developing to unify and "agentify" the ecosystem components. That one is still pretty agile due to being pre-alpha. I already have a benchmark / reporting module in there (low implementation state). https://biocypher.github.io/biotope/latest/
3. Unsure; related to answer 1. If this is a coverage question, I don't have a good enough picture of the landscape.
4. Same answer; although I would just start with the most basic check and align with typical FAIR framework questions. A bit of a chicken-and-egg problem, so any suite of checks is probably fine to start with as long as we stay flexible in adapting to feedback.

If you're trying to avoid a standalone tool, maybe check out biotope and see if this could be a home for it. The only caveat would be that biotope (or biocypher) would limit the reporting to our ecosystem mainly, and reporting for other formalisms would not be sensible to build here; for that purpose, a standalone tool would indeed be more sensible.

Hope that covers everything, happy to discuss further.

[slobentanzer]

@ehsanestaji FYI I have migrated your issue to the organisation discussions.

[ehsanestaji]

@slobentanzer Thanks. I’ll take a closer look at Biotope and the existing benchmark/reporting module before proposing anything concrete. My plan is to come back with a small analysis and, if it still makes sense, one narrow first step around a basic FAIR-style report/check rather than a standalone tool. I’ll follow up soon with something more specific.