Check for IDSPACE conflicts on new ontology submission

[jamesaoverton]

#1703 indicates a larger problem with namespace conflicts.

New projects should be encouraged (required?) to check for conflicts. Checking https://bioregistry.io/ would be the easiest and most effective place to look currently. The easiest thing would be to update our instructions. Better would be an automated check, maybe in https://github.com/OBOFoundry/obo-nor.github.io.

Our current documentation https://obofoundry.org/id-policy.html#allocating-idspaces points to http://identifiers.org/, but it does not include "EPSO" and would not have helped in this case.

[cthoyt]

Thanks for writing this up, James. As we've noted on biopragmatics/bioregistry#273, the Bioregistry does not a likely will never consume the full BioPortal, so we should consider the other aspect of whether the OBO Foundry would want to fully respect the prefixes minted in the BioPortal or not (e.g., there can be and in some cases already are nonsensical overlaps/conflicts with high quality resources in the OBO Foundry, etc.)

[matentzn]

@cthoyt Do you have a list of conflicts between BioPortal and OBO by any chance?

[cthoyt]

Cross post of biopragmatics/bioregistry#273 (comment):

The conflicts that I've curated manually are all in this file https://github.com/biopragmatics/bioregistry/blob/main/src/bioregistry/data/mismatch.json

The curation sheet for BioPortal (which represents all unaligned prefixes) is https://github.com/biopragmatics/bioregistry/blob/main/src/bioregistry/data/external/bioportal/curation.tsv. Because the BioPortal API does not let you get direct access to most of the metadata for each entry, unfortunately the only thing that's represented in this sheet is the BioPortal prefix and BioPortal name. This makes it awfully tricky/time-consuming to do untargeted curation

[matentzn]

@nataled for now we need to document clearly that a new request must not conflict with anything in bioregistry, BioPortal. I can

• Modify thew new request template with a request to manually check bioregistry & BioPortal

before we:

• Implement automated checks as part of the dashboard

[nataled]

https://obofoundry.org/id-policy.html#allocating-idspaces currently indicates to check with identifiers.org. Please confirm that this should be replaced with bioregistry and bioportal. Alternatively, the latter two can be added.

[cthoyt]

See previous discussion on #1519

[nataled]

Clear as mud! ;)

So...
identifiers.org
bioregistry.io
bioportal.org
obofoundry.org
n2t.net
...and...? (I saw several others mentioned, without URLs, like PrefixCommons and BioContext)

I'm looking for a definitive list of resources (names and URLs), specifically for the lists themselves (not some upper-level landing page). In other words, the user should be able to go to the link we provide and see the list of prefixes. Failing that, some page that provides a search function.

[cthoyt]

The Bioregistry imports Identifiers.org, OBO Foundry, and N2T as well as many other resources (see here for a full list), so it can be a one-stop shop for most resources. However, it does not import all of BioPortal, so users should check there too.

Web Access

• Landing page with list of all prefixes in Bioregistry: https://bioregistry.io/registry/
• Landing page with list of all prefixes in Bioportal: https://bioportal.bioontology.org/ontologies
• Home page with search function for Bioregistry: https://bioregistry.io
• Home page with search function for Bioportal: https://bioportal.bioontology.org/
• Search API for Bioregistry: https://bioregistry.io/api/search?q=<query goes here>
• Search API for Bioportal: not sure if this exists

Data Dumps

Bioregistry also has several full dumps

• TSV
• JSON
• YAML
• RDF

for potential contributors who want to access this information programmatically. These are all updated on a nightly basis.

Bioportal doesn't offer any first-party data dumps, but the Bioregistry generates one nightly at https://github.com/biopragmatics/bioregistry/blob/main/src/bioregistry/data/external/bioportal/raw.json

Programmatic Access

Programmatic way to check if something is in the Bioregistry:

import bioregistry

query = "EPSO"
available_in_bioregistry = bioregistry.normalize_prefix(query) is None

Programmatic way to check if something is in BioPortal:

from bioregistry.external.bioportal import get_bioportal

query = "EPSO"
bioportal_dict = get_bioportal()
available_in_bioportal = query not in bioportal_dict

[nataled]

Perfect, thanks!

[cthoyt]

@nataled I updated my comment above with more information that might be more actionable. Feel free to reuse part or all of it

[nataled]

I have updated the documentation (which is outside the scope of this ticket). Please see this page: https://obofoundry.org/id-policy.html and look for the section Allocating IDSPACEs and the subsection Guidelines for selecting an IDSPACE

[nataled]

@matentzn note that my changes should satisfy your first checkbox regarding updating the instructions (the template itself already points to the document I just revised; I see no need to add text to the template itself since that will just duplicate the information).

[matentzn]

This all looks great. I made an issue at the OBO dashboard to implement @cthoyt checker:
OBOFoundry/OBO-Dashboard#59

Thank you both for dealing with this! Are there any remaining action items here?