October 2021 release

• Important: The way we install python packages has changed significantly: we are now using fixed version dependencies (issue). If there are problems with the versions of packages we are using, please let us know immediately.
• We are now using multi-stage builds in ODK for docker. The obolibrary/odklite container is considerably lighter (smaller) than the normal odkfull container you have been using so far, and it should be sufficient for most ODK pipelines.
• Mirrors are now downloaded with CURL instead of ROBOT, and configurable with mirror_retry_download and mirror_max_time_download in the import_group section of the ODK config file (PR). Note that the default mirror_max_time_download is 200 sec (a bit more than 3 minutes), which may be tight for some huge ontologies.
• odkfull now includes Soufflé.
• For those who are using DOSDP patterns we re-introduced the pattern schema check even if pattern generation is skipped (it is very fast).
• Created a simple way to cite ODK from within GitHub, using the CITATION.cff file. If you go to https://github.com/INCATools/ontology-development-kit, you will now see a "cite this repo" section under the About section.
• Changed the default README.md to include a better reference to ODK, the correct ontology description text and the correct edit file extension.
• Added a GitHub action to deploy the ODK-based mkdocs documentation (issue).
• Created a page for frequently used ODK commands.
• Bugfixes:
  • project repo name was not read correctly during make update_repo causing it to be named "False" (commit)
  • Fixed a bug where mirrors were not considered precious
  • Fixed a bug where where the report directories where not created when running a sparql report command (commit)
  • Fixed a bug where report command incorrectly did not include components in the check (issue)
  • Fixed a bug where myont-odk.yaml was not created when using command line mode during seeding (issue)
  • Added some tests for external tools (issue) to ODK built process
  • Konclude now works on the arm64 variant.

June 2020 release

• Switched to a more up-to-date base image (ubuntu-20.04, pull)
• Lots of technical changes on how to manage releases on dockerhub, especially multi-arch (same image should now work on M1, i.e. arm64 and amd64 machines, pull)
• Pip install operations now using python3 -m pip (pull)
• Introduced a new option (remove_owl_nothing) in repo config to remove mentions of owl:Nothing in releases
• New python packages: linkml, kgx and funowl
• Moved a comprehensive test suite to GitHub actions (pull)
• Updated the documentation on how to create a new repo with the ODK
• Updated Repo README with more docs, a list of significant external contributors (please just let me know if anyone else needs to be added), core team etc.
• Bugfixes:
  • Unnecessary use of .FORCE in custom components removed (pull)
  • Fix another bug with OWL DL profile validation (pull)
  • Changed the -g/--skipgit option be be a flag (pull)
  • Some fixes to how subsets are handled (pull)

May 6 2021 release

v1.2.28 (6 May 2021)

• New DOSDP tools version 0.17
• Major: we developed an Apple M1 compatible ODK image! Thanks to @gouttegd who did all the hard work. Super excited about this! The new image can be found here: https://hub.docker.com/r/obotools/odkfull_m1/
• Some major fixes to the automatic documentation generator, see for example https://pato-ontology.github.io/pato/. Please get in touch if you want help setting up your own awesome documentation site!
• A bug fix in the OWL2DL validation pipeline for testing
• A bug fix for the subset injection during Module creation, see here
• Minor fixes to GitHub instructions (switching wordings from master to main)

2021-04-04 release

v1.2.27 (4 April 2021)

• revived odklite image with a minimum install (smaller in size than odkfull, with just robot and owltools!)
• revived robot image with just robot installed
• Added Jena 3.12.0 to odkfull image
• Added module configs (see docs)
• added a complete new documentation system for ontologies using mkdocs (see docs)
• added OWL2 DL profile checking (see here for example)
• Added sqlite3, dos2unix, and aha to odkfull image
• Added a use_custom_import_module feature (see docs for example)
• Added ability to add completely customised release artefacts (see Uberon repo for example)
• Revised update mechanism. Please run the make update_repo multiple times if you encounter problems and to ensure that all changes are picked up.

v.1.2.26 release (February 2020)

• New versions:
  • fastobo validator to new version (pull
  • ROBOT 1.8.1 (lots of new changes, see: Changelog)
  • DOSDPTOOLS 0.16 (lots of speed-up for bulk pattern generation)
• New features:
  • new python dependencies (cogs, a tool to directly manage tsv files in your repo on Google sheets)
  • stable serialisation order for JSON files using jq's walk function. -> this decreases the size of the diff for git!
  • Some improvements to logging when seeding a new repo, to make it easier to find errors
  • A new method to validate the id-ranges file can be invoked using sh run.sh make validate_idranges (after update to latest ODK repo)
  • modules are now annotated with a dc:source annotation to their original ontology (version)
• New configuration options:
  • module_type (example slme, example minimal, example mireot). Direct support for MIREOT and a new module type, minimal.
  • To encourage stable versions and releases, ODK, by default, merges imports into the release files. Previously, we continued to release the imports as well - which we do not recommend anymore. If you still wish to release your imports as usual, you can set a flag release_imports in the import_group section of your makefile (see example).
  • the same as the above applies for reports (see example)
  • The custom sparql checks, and the custom sparql exports, are now directly configurable
    • custom_sparql_checks : Chose which additional sparql checks you want to run. The related sparql query must be named CHECKNAME-violation.sparql, and be placed in the src/sparql directory (see example)
    • custom_sparql_exports : Chose which additional sparql checks you want to run. The related sparql query must be named CHECKNAME-violation.sparql, and be placed in the src/sparql directory (see example)
    • custom_sparql_exports : Chose which custom reports to generate. The related sparql query must be named CHECKNAME.sparql, and be placed in the src/sparql directory (see example)
  • git_main_branch : The main branch for your repo, default main, or (now discouraged, previously) master.
  • ci: continuous integration defaults; currently available: travis, github_actions
  • create_obo_metadata: This is mainly for new OBO ontologies. If true, OBO Markdown and PURL configs are created.
  • export_project_yaml: Default False. If set to True, project.yaml is created in the top level of the repo.
• Removed a few files from the standard config. This is all part of an effort to slimming down the ODK to the least number of necessary files checked into version control: src/ontology/Dockerfile,src/ontology/patterns.sh, src/ontology/release.sh, src/ontology/test.sh, and some temporary files. The patterns directory and all its contents only appear now when use_dosdps=TRUE.
• Technical:
  • Refactored ODK Dockerfile (merged some layers)
  • added jq 1.6 which is not available via apt-get (yet).
  • added sssom python package, but its in so alpha-alpha state, it should be used with caution

February 2021 ODK 1.2.26 release (amended)

Some hotfixes and a features added to v.1.2.26 release (all pertaining to ODK repo template, not the tools)

• Hotfixes:
  • The new mireot module technique was buggy and is therefore removed again. Sorry; we will try again next time. You can still use the custom option to implement mireot yourself!
  • A change in the way imports were processed introduced a very high memory footprint for large ontologies and slowed stuff down. If you do not have a lot of memory (and time!) available, you should use the following new flags: is_large and use_gzipped. is_large: TRUE introduces a special handling for the ontology that is faster and consumes less memory when creating an import. Using use_gzipped will try to download the ontology from its gzipped location. Make sure its actually there (we know its the case for chebi and pr at least)!

import_group:
  products: 
    - id: pr
      use_gzipped: TRUE
      is_large: TRUE
    - id: chebi
      use_gzipped: TRUE
      is_large: TRUE

• An irrelevant file (keeprelations.txt) was still generated even if needed when seeding a new repo.
• Module type STAR was accidentally hard coded default for slme. Now changed to BOT as it was.
• CI configs where not correctly copied by update routine. Now it does. Note for the changes to be picked up, you need to run sh run.sh make update_repo twice (once for updating the update script itself)!
• Geeky (but necessary) all phony make goals are now correctly declared as .PHONY.
• Some last minute features:
  • In new repos, the README.md is now generated with the correct, appropriate banners.
  • We now have a new feature, custom_makefile_header, that allows injecting a custom header into the Makefile. Most mortals wont need this, but this is how it goes:

custom_makefile_header: |
  ### Workflow
  #
  # Tasks to edit and release OMRSE.
  #
  # #### Edit
  #
  # 1. [Prepare release](prepare_release)
  # 2. [Refresh imports](all_imports)
  # 3. [Update repo to latest ODK](update_repo)

• all features and fixes here: #397

2020-11-18 November release (Hotfix)

updated v1.2.25 (18 November 2020)

• Updated ROBOT to new version 1.7.2, which includes some hotfixes for ROBOT report and update to whelk 1.0.4
• Fixed a bug (#376) that prevented certain things (like imports and pattern generation processes) to be printed when running the Makefile.

2020-11-06 (1.2.24) ODK Release

• Updated ROBOT to new version 1.7.1
• Added the (highly experimental) ability to ODK to run OBO dashboard (see instructions and examples).
• Added more python packages to ODK, see requirements.txt.
• Added a new set of configurations for ROBOT report. WARNING:
  report_fail_on option is now deprecated in favour of a new block of options:

robot_report:
  use_labels: TRUE
  fail_on: None
  custom_profile: TRUE
  report_on:
    - .owl
    - .obo
    - edit

• use_labels: allows switching labels on and off in the ROBOT report, see here
• fail_on: is the old report_fail_on option, see here.
• custom_profile: allows switching on custom profiles, see here
• report_on allows specifying which files to run the report over (for example hp.owl, hp.obo, hp-edit.owl).

2020-08-06 (1.2.23) ODK Release

Changes:

• New ROBOT (v 1.7.0)
• New owltools (v. 2020-04-06)
• Added SWI Prolog and C. Mungall's sparqlprog tools
• Release artefacts are now annotated with owl:versionInfo
• run.sh now got the JAVA_OPTS parameter which is necessary to set the maximum memory allowed for owltools
• Bug fixes:
  • base-modules are now correctly annotated with resource references rather than strings (issue)
  • The ODK_VERSION variable in the Makefile was misnamed, which causes a circularity with updating it correctly when the Makefile is updated with a new ODK. This is fixed, but for the fix to kick in, you have to run sh run.sh make update_repo twice this time!

Please get the latest ODK from dockerhub.

2020-08-06 (1.2.23) ODK Release (UPDATED)

Changes:

Original changes:

• New ROBOT (v 1.7.0)
• New owltools (v. 2020-04-06)
• Added SWI Prolog and C. Mungall's sparqlprog tools
• Release artefacts are now annotated with owl:versionInfo
• run.sh now got the JAVA_OPTS parameter which is necessary to set the maximum memory allowed for owltools
• Bug fixes:
  • base-modules are now correctly annotated with resource references rather than strings (issue)
  • The ODK_VERSION variable in the Makefile was misnamed, which causes a circularity with updating it correctly when the Makefile is updated with a new ODK. This is fixed, but for the fix to kick in, you have to run sh run.sh make update_repo twice this time!

Please get the latest ODK from dockerhub.

Updated version:

• Added new version of fastobo (v0.3.0)
• Added rdflib to Python environment
• Added xlsx2csv command line tool