Skip to content

smarie/pyLODE

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

209 Commits
img
img
 
 
pylode
pylode
 
 
tests
tests
 
 
.gitignore
.gitignore
 
 
LICENSE
LICENSE
 
 
README.rst
README.rst
 
 
requirements.dev.txt
requirements.dev.txt
 
 
requirements.txt
requirements.txt
 
 
setup.cfg
setup.cfg
 
 
setup.py
setup.py
 
 

Repository files navigation

img/pyLODE-250.png

pyLODE

An OWL ontology documentation tool using Python and templating, based on LODE.

In addition to making human-readable forms of ontologies/taxonomies, pyLODE encourages ontology annotation best practice by only producing good results for well documented inputs! pyLODe defines what it considers w'well documented' in sections below, such as Profiles & What pyLODE understands.

Contents

  1. Quick Intro
  2. Examples
  3. Installation
  4. Use
  5. What pyLODE understands
  6. Profiles
  7. Differences from LODE
  8. Releases roadmap
  9. License
  10. Citation
  11. Collaboration
  12. Contacts

Quick Intro

The Live OWL Documentation Environment tool (LODE) is a well-known (in Semantic Web circles) Java & XSLT-based tool used to generate human-readable HTML documents for OWL and RDF ontologies. That tool is now a bit dated (old-style HTML, use of older technologies like XSLT) and it's (online version) is not always online.

This tool is a complete re-implementation of LODE's functionality using Python and Python's RDF manipulation module, rdflib. An ontology to be documented is parsed and inspected using rdflib and HTML or Markdown is generated using basic Python scripting and Python's Jinja2 templating.

The tool can be run as in these ways:

Examples

pyLODE has been tested with all of the 30+ ontologies in pylode/examples/ and we are trying to ensure it captures all of their annotations. For each example, there is the original RDF file and the corresponding output, in HTML & Markdown.

For example, Epimorphic's's Registry Ontology is:

  • reg.ttl - source file
  • reg.html - HTML output
  • reg.md - Markdown output

Another, the Australian Government's Records Interoperability Framework (AGRIF) Ontology:

You can build all of the example outputs locally by running pylode/examples/_make_examples.py which also serves as a good demonstration of calling pyLODE from a Python file.

Ontologies online using pyLODE:

See pairs of RDF & HTML files in the pylode/examples/ directory for other, preprocessed examples.

Installation

This tool can be used either as a command line utility (Linux, Mac or Windows, see below) or as a Python module in other Python code. It can also be used via an online API. This repo contains executable files for Mac & Windows (soon Linux!) that you can use without any installation too.

The most important dependency to get correct when using this as a Python script of a command line program is the package rdflib which must be v5.0.0 or greater (see requirements.txt).

Python

Do this to use pyLODE as a Python command line program.

This tool is available on PyPI, the Python Package Index, at https://pypi.org/project/pyLODE/ and can be installed for use as a Python module via pip:

pip install pylode

For desktop command line use, just clone this repository and either use cli.py as per the command line instructions below or use makedocco.py as a Python script directly.

Use

pyLODE presents natively as a Python command-line utility, pylode/cli.py and there are also a BASH, Windows & Mac OS options available for command line use:

Additionally, there is a Falcon framework local HTTP server option.

All use the same command line arguments.

Command line arguments

These are the command line arguments to run pyLODE as a BASH or Python script on Linux, Mac etc. or via the Windows executable:

  • -i or --inputfile, required if -u not used
    • The RDF ontology file you wish to generate HTML for Must be in either Turtle, RDF/XML, JSON-LD or N-Triples formats indicated by the file type extensions .rdf, .owl, .ttl, .n3, .nt, .json respectively
  • -u or --url, required if -i not used
    • The RDF ontology you wish to generate HTML for, online. Must be an absolute URL that can be resolved to RDF, preferably via Content Negotiation.
  • -c or --css, optional, default 'false'
    • Whether (true) or not (false) to copy the default CSS file to the output directory.
  • -o or --outputfile, optional
    • A name you wish to assign to the output file. Will be postfixed with .html or .md. If not specified, the name of the input file or last segment of RDF URI will be used, + .html/.md.
  • -f or --outputformat, optional, default 'html'
    • The output format of the documentation. 'html' or 'md' accepted.
  • -p or --profile, optional, default 'owl'
    • The profile (specification) for ontology documentation used. This has been "owl" (for OWL Ontology) only until the recent introduction of "skosp" (according to the Simple Knowledge Organization System (SKOS)). See -lp for all profiles supported.
  • -lp or --listprofiles, optional, no arguments
    • Lists all the profiles (specifications) for ontology documentation supported by pyLODE

Example call

This call to the BASH script in pylode/bin/ will create an HTML document for an ontology called placenames.html and save it with a basic CSS file into pylode/output_files/:

./pylode -i ../example/prof.ttl --css true

Online API

An online API to access pyLODE is now available in test mode at https://kurrawong.net/pylode-online.

Local server - Falcon

You can run pyLODE using your own, local, HTTP server like this:

cd pylode && gunicorn --chdir /path/to/pyLODE/pylode server:api

Then, in another terminal:

curl localhost:8000/lode?url=http://sweetontology.net/sweetAll.ttl

Windows - create EXE from source

A Windows binary (.exe file) is included in this repository in the pylode/bin/ folder which has been produced via Pyinstaller. Additionally, Pyinstaller can be used to create a new executable. These Windows executables have the same characteristics as the Linux/Mac CLI program.

Pyinstaller uses a .spec file to make the binary and that is included in this repository: pylode-cli.spec.

See the PyInstaller installation guide for info on how to install PyInstaller for Windows.

Once you have Pyinstaller, use pyinstaller to generate the pyLODE.exe CLI file like so:

cd pylode
pyinstaller pylode-cli.spec

This will output pylode.exe in the dist directory in pylode. The .exe file in bin/ is just the latest copy of this.

You can now run the pyLODE Command Line utility via pylode.exe. See above for the pyLODE command line util usage instructions.

Mac OS - create APP from source

As per instructions for PyInstaller use on Windows, just do it on a Mac! The result is a pylode file which is renamed bin/pylode.app.

What pyLODE understands

Annotations

pyLODE understands the following ontology constructs:

  • ontology metadata
    • imports - owl:imports
    • title - rdfs:label or skos:prefLabel or dct:title or dc:title
    • description - rdfs:comment or skos:definition or dct:description or dc:description
      • Markdown is supported
    • historyNote - skos:historyNote
      • Markdown is supported
    • version URI - owl:versionIRI as a URI
    • version info - owl:versionInfo as a string
      • preferred namespace prefix - vann:preferredNamespacePrefix as a token
      • preferred namespace URI - vann:preferredNamespaceUri as a URI
    • agents: publishers, creators, contributors
      • see Agent Formatting below for details
      • see the pylode/examples/ directory for examples!
    • dates: created, modified, issued - dct:created etc., all as xsd:date or xsd:dateTime datatype properties
    • rights: license - dct:license as a URI & rights - dct:rights as a string
    • code respository - schema:codeRepository as a URI
    • source - dcterms:source as a URI or text
  • classes
    • per rdfs:Class or owl:Class
    • title - rdfs:label or skos:prefLabel or dct:title
    • description - rdf:comment or skos:definition or dct:description as a string or using Markdown or HTML
    • usage note - a skos:scopeNote string
    • example - a skos:example string containing RDF
    • super classes - by declaring a class to be owl:subClassOf something
    • sub classes - pyLODE will work these out itself
    • restrictions - by declaring a class to be owl:subClassOf of an owl:Restriction with any of the normal cardinality or property existence etc. restrictions
    • in domain/range of - pyLODE will auto-calculate these
  • properties
    • per owl:ObjectProperty, owl:DatatypeProperty or owl:AnnotationProperty
    • title - rdfs:label or skos:prefLabel or dct:title
    • description - rdf:comment or skos:definition or dct:description
    • usage note - a skos:scopeNote string
    • example - a skos:example string containing RDF
    • super properties - by declaring a class to be owl:subPropertyOf something
    • sub properties - pyLODE will work these out itself
    • equivalent properties - by declaring a class to be owl:equivalentProperty something
    • inverse of - by declaring a class to be owl:inverseOf something
    • domains - rdfs:domain or schema:domainIncludes
    • ranges - rdfs:range or schema:rangeIncludes
  • namespaces
    • pyLODE will honour any namespace prefixes you set and look up others in http://prefix.cc
    • it will either read your ontology's default/base URI in annotations or guess it using a number of methods
  • named individuals
    • coming!

Agents

Agents, individual persons or organisations, should be associated with ontologies to indicate authors, creators, publishers etc. There are 2 ways to do this that pyLODE understands: datatype & object type.

Datatype - not preferred

A simple literal value for an agent that a human can read but not a machine can't understand:

  • <ONTOLOGY_URI> dc:creator "AGENT NAME" .
    • the range value is a string literal, either string typed (^^xsd:string) or language typed (@en or @de)
    • the following Dublin Core Elements 1.1 properties may be used:
      • dc:contributor
      • dc:creator
      • dc:publisher
    • the following schema.org properties may be used:
      • schema:author
      • schema:contributor
      • schema:creator
      • schema:editor
      • schema:funder
      • schema:publisher
      • schema:translator
<ontology_x>
    dc:creator "Nicholas J. Car" ;
Object type - preferred

An RDF object is used for the agent and can contain multiple details. A Blank Node or a URI can be used. Best case, a persistent agent URI!

img/contributor-object.png
  • <ONTOLOGY_URI> dct:creator [...] .

or

  • <ONTOLOGY_URI> dct:creator <SOME_URI> .
    • the range value is a Blank Node or a URI of type:
      • schema:Person
      • schema:Organization
      • foaf:Person
      • foaf:Organization
    • the properties of the Blank Node or the URI are as below
    • the following Dublin Core Terms properties may be used:
      • dct:contributor
      • dct:creator
      • dct:publisher
      • dct:rightsHolder
    • the following schema.org properties may be used:
      • schema:author
      • schema:contributor
      • schema:creator
      • schema:editor
      • schema:funder
      • schema:publisher
      • schema:translator
    • the following FOAF properties may be used:
      • foaf:maker

e.g. (Blank Node):

<ontology_x>
    schema:editor [
        a schema:Organization ;
        ...
    ] ;

or (URI):

<ontology_x>
    schema:editor <https://orcid.org/0000-0002-8742-7730> ;
    ...

<https://orcid.org/0000-0002-8742-7730>
    a foaf:Person ;
    ...
Agent datatype properties
  • foaf:name / schema:name
  • foaf:mbox / schema:email
  • foaf:homepage / schema:url
  • schema:identifier

e.g.:

<ontology_x>
    dct:creator [
        schema:name "Nicholas J. Car" ;
        schema:identifier <http://orcid.org/0000-0002-8742-7730> ;
        schema:email <mailto:nicholas.car@surroundaustralia.com> ;
    ] ;
Linking a Person to an Organization

Use schema:member, schema:affiliation (there is no FOAF Person -> Group/Org property):

e.g.:

<ontology_x>
    dct:creator [
        schema:name "Nicholas J. Car" ;
        schema:identifier <http://orcid.org/0000-0002-8742-7730> ;
        schema:email <mailto:nicholas.car@surroundaustralia.com> ;
        schema:affiliation [
            schema:name "SURROUND Australia Pty Ltd" ;
            schema:url <https://surroundaustralia.com> ;
        ] ;
    ] ;

Provenance

Ontology Source

The ontology's HTML representation linking back to the RDF: generated automatically

img/source.png
Code Repositories

Indicating to readers where the 'live' version of the ontology is managed:

img/code-repository.png

Code repositories that house an ontology can be indicated either using schema.org's codeRepository property or a combination of the Description of a Project and PROV:

@prefix schema: <https://schema.org/> .

<ONTOLOGY_URI>
    schema:codeRepository <REPO_URI> ;
    ...

or

@prefix doap: <http://usefulinc.com/ns/doap#> .
@prefix prov: <http://www.w3.org/ns/prov#> .

<ONTOLOGY_URI>
    prov:wasGeneratedBy [
        a doap:Project , prov:Activity ;
        doap:repository <REPO_URI>
    ]
    ...

e.g., for the ontology version on ISO 19160-1:

<http://linked.data.gov.au/def/iso19160-1-address>
    prov:wasGeneratedBy [
        a doap:Project , prov:Activity ;
        doap:repository <https://github.com/AGLDWG/iso19160-1-address-ont>
    ] ;
    ...

Styling

This tool generates HTML that is shamelessly similar to LODE's styling. That's because we want things to look familiar and LODE's outputs look great. The Markdown's pretty vanilla.

Also, pyLODE generates and uses only static HTML + CSS, no JavaScript, live loading Google Fonts etc. This is to ensure that all you need for nice display is within a couple of static, easy to use and maintain, files. Prevents documentation breaking over time.

Feel free to extend your styling with your own CSS.

Profiles

pyLODE can document ontologies and other taxonomies according to different profiles which are specifications. The basic, default, profile is pyLODE's OWL Profile, which means documentation is generated according to OWL properties and classes and the various annotation properties listed here in the What pyLODE understands section.

pyLODE can tell you what profiles it supports: just run ~$ pylode -lp ("list profiles") or, if calling from Python:

m = MakeDocco(input_data_file="examples/data-access-rights.ttl", profile="skosp")
print(m.list_profiles())

Supported Profiles

Currently pyLODE supports its OWL profile, as described above, and a profile of SKOS. For full details of what the profiles include, see the profiles' definitions at:

Token URI
owlp https://w3id.org/profile/pylode-owl
skosp https://w3id.org/profile/pylode-skos

Transformation by Profile

You can, of course, document an OWL ontology using the owlp profile or a SKOS taxonomy using the skosp profile however, you can also document an OWL ontology using the skosp profile! This is because SKOS is conceptually a subset of OWL - whatever you can express in SKOS you can express in OWL.

pyLODE performs an OWL > SKOS transformation on OWL ontologies to produce a taxonomy document. The following conversions are made:

  • owl:Ontology > skos:ConceptScheme
    • and all the ontology metadata is used with the ConceptScheme
  • owl:Class > skos:Concept
    • and other class annotation properties used with Concept
  • owl:subClassOf > skos:broader
    • and the inverses, skos:narrower

To see the full list of transformations, see the function _expand_graph_for_skos() in makedocco.py.

Examples of a small taxonomies documented using the skosp profile are:

An example of a large one:

An example of a skosp-documented OWL ontology and the corresponding owlp original is AGRIF:

Differences from LODE

  • command line access
    • you can use this on your own desktop so you don't need me to maintain a live service for use
  • use of more modern & simpler HTML
  • catering for a wider range of ontology options such as:
    • schema.org domainIncludes & rangeIncludes for properties
  • better Agent linking
    • foaf:Agent or schema:Person objects for creators, contributors & publishers
    • you can still use simple string peoperties like dc:contributor "Nicholas J. Car" too if you really must!
<ontology_x>
    dct:creator [
        sdo:name "Nicholas J. Car" ;
        sdo:identifier <http://orcid.org/0000-0002-8742-7730> ;
    ] ;
  • smarter CURIES
    • pyLODE caches and looks up well-known prefixes to make more/better CURIES
    • it tries to be smart with CURIE presentation by CURIE-ising all URIs it finds, rather than printing them
  • active development
    • this software is in use and will be improved for the foreseeable future so we will cater for more and more things
    • recent ontology documentation initiatives such as the MOD Ontology will be handled, if requested

Releases Roadmap

pyLODE is under continual and constant development. The current developers have a roadmap for enhancements in mind, which is given here, however, since this is an open source project, new developers may join the pyLODE dev community and change/add development priorities.

The current release, as of May 2020, is 2.4.

pyLODE Release Schedule
Version Date Description
3.0 June 2020 Will include pre-testing inputs with SHACL
2.4 27 May 2020 Small improvements over 2.0
2.0 18 Apr 2020 Includes multiple profiles - OWP & SKOSP
1.0 15 Dec 2019 Initial working release

Release notes

3.0 - expected

Expected to handle

  • pre-documentation graph shape testing using SHACL
    • you will be able to see what pyLODE-recommended annotation and design patterns your inputs do/don't handle
  • "modp", a documentation profile based on the MOD Ontology

2.0 - current

  • handles Named Individuals in OWL ontologies
  • implements "owlp" & "skosp" documentation profiles for OWL, SKOS and OWL-as-SKOS results

1.0 - previous

Initial pyLODE release. Generated HTML documentation for OWL ontologies, missed quite a few expected ontology elements, such as Named Individuals.

License

This code is licensed using the GPL v3 licence. See the LICENSE file for the deed. Note Citation below though for attribution.

Citation

If you use pyLODE, please leave the pyLODE logo with a hyperlink back here in the top left of published HTML pages.

Collaboration

The maintainers welcome any collaboration.

If you have suggestions, please email the contacts below or leave Issues in this repository's Issue tracker.

But the very best thing you could do is create a Pull Request for us to action!

Contacts

Author:
Nicholas Car
Data System Architect

About

An OWL ontology documentation tool using Python and templating, based on LODE

Resources

Readme

License

GPL-3.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

21 tags

Packages

 
 
 

Contributors

Languages

  • HTML 94.9%
  • Python 4.9%
  • Other 0.2%