0.6.0

Release notes for pymetadata 0.6.0

We are pleased to release the next version of pymetadata including the
following changes. This is a large release which reorganizes the package, so it
starts with what has to be changed in existing code.

Breaking changes

• python 3.10 support dropped, python >=3.11 is now required
• the generated ontologies moved from pymetadata.metadata to
  pymetadata.ontologies, next to the module which generates them, and the
  pymetadata.metadata package was removed:
  from pymetadata.metadata import SBO becomes
  from pymetadata.ontologies import SBO
• the modules which query web services are collected in the new
  pymetadata.webservices package: chebi, unichem and webservice moved
  there from the top level, ols from pymetadata.ontologies and registry
  from pymetadata.identifiers. miriam moved to pymetadata.core and the
  pymetadata.identifiers package was removed:
  from pymetadata.identifiers.miriam import BQB becomes
  from pymetadata.core.miriam import BQB
• the ontologies are no longer str enums but classes of OntologyTerm
  attributes, see the feature below. A term is still the identifier, i.e.,
  SBO.SIMPLE_CHEMICAL == "SBO_0000247" and serialization are unchanged, and
  the lookups of an enum are kept (len, iteration, in,
  SBO["SBO_0000247"], SBO("SBO:0000247"), get_name, validate). repr
  shows the label now, e.g., <SBO.SBO_0000247: 'simple chemical'>, and
  ENUM_DIR was renamed to ONTOLOGY_DIR
• ECO is no longer packaged: the evidence terms were not used by the package
  and the module was the largest of the generated ones. The ontology can still
  be queried through OLS
• web service responses are cached on disk by default, i.e.,
  pymetadata.CACHE_USE changed from False to True, so repeated lookups of
  the same term do not query OLS, ChEBI or UniChem again. The cache is written
  to pymetadata.CACHE_PATH (~/.cache/pymetadata), deleting it is always
  safe, and pymetadata.CACHE_USE = False restores the previous behaviour of
  querying the services every time
• annotations are normalized to identifiers.org compact identifiers of the form
  https://identifiers.org/<prefix>:<accession> (#71); resource_normalized
  no longer emits the deprecated https://identifiers.org/<prefix>/<accession>
  form for namespaces which are not embedded in the LUI, i.e.,
  NCIT:C180619 now normalizes to https://identifiers.org/ncit:C180619
  instead of https://identifiers.org/ncit/C180619
• pymetadata no longer configures logging: it adds no handlers and sets no
  levels, so messages are formatted and filtered by the application.
  log.enable_rich_logging() turns the rich output on for scripts and
  examples. Importing the package no longer replaces the interpreter
  displayhook via rich.pretty.install()
• log.get_logger() removed, it only returned logging.getLogger(name). Code
  which called it changes from log.get_logger(__name__) to
  logging.getLogger(__name__); pymetadata.log keeps enable_rich_logging()
  and PACKAGE_LOGGER
• pronto is an optional dependency now, installed with
  pip install pymetadata[ontology]. It is only used to read ontologies and
  generate the modules, not to use the shipped terms, which reduces an
  installation from 37 MB to 22 MB and removes the compiled fastobo and
  networkx dependencies
• pymetadata.ontologies.ontology renamed to
  pymetadata.ontologies._ontology_builder: the module downloads the
  ontologies and generates the modules, it is internal tooling for maintainers
  and no longer part of the public API or the API reference. Regeneration runs
  with python -m pymetadata.ontologies._ontology_builder
• the examples moved out of the package to examples/ in the repository, the
  _example suffix is gone and the COMBINE archive examples are collected in
  examples/omex/ together with the archives they use. They run as scripts,
  e.g., python examples/omex/omex.py, instead of
  python -m pymetadata.examples.omex_example, and the example data is no
  longer shipped in the distribution. A pbpko example was added
• minimum versions of the dependencies raised to rich>=15.0.0,
  requests>=2.34.2 and pydantic>=2.13.5. They are widely used packages and
  the current releases are what pymetadata is tested against

Features

• ontology terms carry the information of the ontology release: a term of
  SBO, KISAO or PBPKO provides label, definition, synonyms,
  deprecated, curie and url, and get_term resolves a term given as a
  string. The terms are declared with their definition as docstring, so an
  editor completing SBO. shows what a term means
• PBPKO (PBPK ontology) added as a first class ontology (#70), i.e., PBPKO is
  available from pymetadata.ontologies and the ontology is updated with the
  others. SBO and KISAO were regenerated from the current ontology releases
• cached responses expire, instead of being kept forever: the OLS, ChEBI and
  UniChem responses after 30 days (CACHE_DURATION_ONTOLOGY), since they
  describe the terms of an ontology release, the identifiers.org registry after
  24 hours (CACHE_DURATION_REGISTRY), since namespaces and their patterns are
  added and corrected continuously
• the queries survive an outage: if content has to be refreshed but the service
  cannot be reached, the outdated cache is used and a warning is logged instead
  of the query failing, so annotating keeps working offline. Only a query which
  was never cached fails
• web service queries are retried on the transient error responses (429, 500,
  502, 503, 504) and time out after 30s. The UniChem, ChEBI, OLS and
  identifiers.org queries share a session with these settings
• webservice.get_json() added, which raises a WebserviceError for the
  unreachable service, the error response and the response which is not JSON.
  The query modules use it instead of checking the status code themselves, so a
  ConnectionError or a JSONDecodeError no longer escapes from a ChEBI, OLS,
  UniChem or registry query
• Omex can be used as a context manager, which removes the temporary
  directory with the archive content when the block is left
• importing pymetadata no longer queries the identifiers.org web service. The
  registry and the OLS query object are created on first use and shared, which
  makes import pymetadata.core.annotation work offline and reduces the import
  from ~0.9s to ~0.1s

Fixes

• fix reading a manifest.xml with a single content entry, which raised an
  AttributeError; an archive with a single file is valid OMEX
• fix UniChem queries raising a JSONDecodeError when the service answers with
  an HTML error page instead of JSON; the status code is checked before the
  response is parsed
• fix Ontology storing the read ontology on the class, so reading a second
  ontology replaced the content of the first one
• fix the generation processing the previous term again when a term resolved to
  neither a relationship nor a term of the ontology
• fix the UniChem source cache, which was never shared, so every query object
  retrieved the sources again
• fix the validate methods of the generated ontologies: the isinstance
  check compared against a string instead of the class, which would raise a
  TypeError if that branch were reached
• messages are no longer logged twice when the application configures logging
• namespaces without a prefix are skipped when updating the identifiers.org
  registry instead of ending up in the registry with a None key

Documentation

• documentation restructured (#72): the sources are in docs/, nothing
  rendered is committed any more, and the site is built and published to
  https://matthiaskoenig.github.io/pymetadata by a GitHub workflow on every
  change. New user guides for COMBINE archives and annotations
• documentation migrated from quarto to Zensical, the
  static site generator of the Material for MkDocs authors: markdown sources in
  docs/, configured in zensical.toml, with the API reference rendered from
  the docstrings by mkdocstrings instead of quartodoc. The pages of the site
  moved from /omex.html to /omex/
• API reference extended from 2 to 14 modules, covering annotations,
  qualifiers, cross references, creators, the identifiers.org registry, the
  OLS, ChEBI and UniChem queries, ontology generation and caching. It is
  structured by package, with pymetadata.core and pymetadata.webservices as
  subsections. All docstrings converted to google style, so arguments, return
  values and errors are rendered instead of appearing as raw text
• COMBINE archive documentation and examples improved: the omex API reference
  documents arguments, return values, exceptions and examples for every public
  method, with the OMEX publications cited; the runnable examples were
  rewritten and no longer write into the package
• the HARMONY2026 presentation is no longer published with the documentation,
  its sources remain in presentations/

Development

• mypy replaced by ty for type checking (#69); ty runs in CI, in tox r -e ty
  and as a pre-commit hook. All # type: ignore comments are gone
• type annotations modernized to dict, list and X | None, mutable class
  attributes marked as ClassVar
• more ruff rules (import sorting, docstrings, pyupgrade, bugbear,
  comprehensions, simplifications, returns) and additional pre-commit hooks
  (toml, json, case conflicts, line endings, forgotten debug statements)
• the generated modules are written with plain python instead of a jinja2
  template, so jinja2 is gone from the ontology extra, which now only needs
  pronto, and the template in resources/templates was removed. Regeneration
  downloads only the packaged ontologies instead of all ontologies of
  ontology_files
• xmltodict dependency dropped, the manifest is parsed with the standard
  library; lxml and depinfo removed as well, both unused
• the project urls reach the metadata now: the table was written as
  [project_urls] instead of [project.urls], so it was silently ignored and
  the pypi page carried no links at all. Documentation points to the
  documentation site instead of the repository and a Changelog url was added.
  The license metadata follows PEP 639, i.e., license = "MIT" with
  license-files, instead of the deprecated License :: classifier
• root configuration cleaned up: .gitignore lost the entries of tools which
  are not used, the unused pytest-raises dependency and its raises marker
  are gone, as is the install tox environment which nothing referenced, and
  the pytest configuration moved from tox.ini to [tool.pytest.ini_options]
  in pyproject.toml
• the draft of the COMBINE archive version 2 moved to the new pymetadata.oven
  package, which collects work in progress: not wired into the package, not
  documented, not part of the public API and not shipped in the release

Your pymetadata team