Ontology test guide

How to test for inconsistent ontologies and unsatisfiable classes

Using Protégé

1. Open the ontology in Protégé
2. Under the Reasoner menu, Select the reasoner you want to use. HermiT is installed by default in Protégé but you can also install plugins for Pellet and FaCT++
3. Start reasoner with Strg-R (Ctrl-R) or with menu Reasoner->Start reasoner. Protégé then will inform you if the ontology is inconsistent and give you the option to get explanations. One common example is when Individuals have conflicting types (are instances of two or more disjoint superclasses).
4. The Class hierarchy will popup and let you see if there is a subclass of owl:Thing called owl:Nothing. Unsatisfiable classes will be subclasses of owl:Nothing. This usually happens when there are mutliple disjoint superclasses.
5. Stop reasoner with menu Reasoner->Stop reasoner

Useful when fixing:

• Synchronise reasoner with menu Reasoner-> Synchronise reasoner instead of stopping and starting again

Using the command line interface

0. If your folder structure does not contain a build folder make one.

There is also the alternative of evaluating the resulting ontology using directly the HermiT reasoner outside of the Protégé environment. To do this, you need to do some setup consisting of three steps. The first is a compatible java installation, you can either use the licenced oracle version or the Open JDK. Secondly you need to download the HermiT Reasoner itself which you can store in the the build folder. And lastly you will need to download the ROBOT 🤖 tool and store it in the same place as HermiT.

Once you have setup your environment you can follow these instructions:

1. Compile the ontology using the command below.This should be run from the root directory of the ontology project. Akey component is the java -jar robot.jar command, the robot.jar should be in the given directory. If you have it somewhere elese refer to that location. If you are in linux or Mac you can alternatively use the make command, in this case the output will appear in the build directory.

java -jar build/robot.jar merge --catalog src/ontology/catalog-v001.xml --input src/ontology/edits/oeo-physical.omn --input src/ontology/edits/oeo-shared.omn --input src/ontology/edits/oeo-social.omn --input src/ontology/edits/oeo-model.omn --input src/ontology/edits/oeo-physical-axioms.owl annotate --ontology-iri http://openenergy-platform.org/ontology/oeo/ --output build/oeo/{replace with version in VERSION}/oeo-full.owl

2. Now you can look for inconsistencies by running the command below. If you get any class that is not owl:Nothing as equivalent class, there is an inconsistency. (look at the point 4 of the Protégé instructions). Windows users might need an extra / after file://

java -jar build/hermit.jar file://{REPLACE WITH THE FULL PATH TO YOUR DIRECTORY}/build/oeo/{replace with version in VERSION}/oeo-full.owl

3. Next we are going to test the ontology against the OWL 2 profiles. We are testing against Full and DL. The report merged-validation.txt will contain information on axioms that are incopatible with the given profile.

java -jar build/robot.jar validate-profile --input build/oeo/{replace with version in VERSION}/oeo-full.owl --profile Full --output merged-validation.txt

4. If all the test before passed you can do more detailed evaluations. Namely test the competency questions. To do so you can also use HermiT. You can do this for each question using the command below. If you want to test them in batch you can follow the instructions of the next section.

java -jar build/hermit.jar --premise file://{REPLACE WITH THE FULL PATH TO YOUR DIRECTORY}/build/oeo/{replace with version in VERSION}/oeo-full.owl --conclusion=file:///{REPLACE WITH THE FULL PATH TO YOUR DIRECTORY}/tests/competency_questions/{replace with competency question omn file}

Testing competency questions in batches

Make sure first that you followed the setup steps in the last section and that the built ontology is in the respective build/oeo/{replace with version in VERSION}/ folder. To use the test infrastructure you will need python in your system, it is recommended that you create a virtual environment with conda, virtualvenv or similar tools. Any version after 3.7 should work properly, if not let us know.

1. Once python is installed, install the requirements from src/scripts/requirements.txt

pip install -r src/scripts/requirements.txt

2. Now you should be able to call pytest to check all the tests, run simply:

pytest

3. You can also run some specific tests adding the --selected flag and the name of the test

pytest  --selected some_test_name

Look at an example output:

(lod_env) C:\Work\04_LOD\repositories\ontology>pytest --selected 004    
=========================================================================== test session starts ============================================================================
platform win32 -- Python 3.9.13, pytest-7.2.0, pluggy-1.0.0
rootdir: C:\Work\04_LOD\repositories\ontology
plugins: harvest-1.10.4
collected 2 items

tests\test_competency_questions.py ..                                                                                                                                 [100%]

============================================================================ 2 passed in 1.40s =============================================================================