Improvements to `report` command #391

dougli1sqrd · 2018-10-30T21:58:40Z

Hello,
I'm attempting to replace the set of robot commands and queries in the geneontology/go-ontology repo that we use to validate changes to the ontology in travis checks and the like with robot report. These are run on the go-edit.obo file in that repository. go-edit.obo imports chebi.

In go-ontology/src/ontology/ I run robot report --input go-edit.obo --output report.tsv --profile profile.txt, and the profile.txt looks like:

ERROR	annotation_whitespace
ERROR	deprecated_class_reference
ERROR	duplicate_definition
WARN	duplicate_exact_synonym
ERROR	duplicate_label_synonym
ERROR	duplicate_label
WARN	duplicate_scoped_synonym
ERROR	equivalent_pair
WARN	invalid_xref
ERROR	label_formatting
ERROR	label_whitespace
INFO	lowercase_definition
ERROR	missing_definition
ERROR	missing_label
WARN	missing_obsolete_label
ERROR	missing_ontology_description
ERROR	missing_ontology_license
ERROR	missing_ontology_title
INFO	missing_superclass
ERROR	misused_obsolete_label
INFO	multiple_asserted_superclasses
ERROR	multiple_definitions
ERROR	multiple_labels
ERROR	file:./../sparql/never-in-taxon-non-IRI-value-violation.sparql

The report.tsv indicates about 2000 missing_label errors on CHEBI terms implying they are in this ontology, but these CHEBI terms are not in the go-edit.obo file, they're imported.

So am I doing something wrong in the way I invoke the report command, or on the file that I'm reporting on? Or is this a bug? I see in the code it's supposed to not pull in all the imports I think? Thanks for your help!

The text was updated successfully, but these errors were encountered:

cmungall · 2018-10-30T22:10:24Z

Good point, this is underspecified at the moment.

There was an analogous issue with the query command: #158
see the fix: #328 and the instructions http://robot.obolibrary.org/query

Should we have equivalent options for report?

However, the defaults may have to be different, because as you point out any ontology that has imports and references classes in the import will produce false positive errors. @jamesaoverton @rctauber how do you feel about having the defaults be different here?

beckyjackson · 2018-10-30T22:21:31Z

I would think that you’re only reporting on your base ontology. That way, issues with other ontologies won’t break the report. That said, it might make sense to have an option for including imports anyway, which would make it easier to report on the edit versions of ontologies who use templates like OBI.

…

On Tue, Oct 30, 2018 at 18:10 Chris Mungall ***@***.***> wrote: Good point, this is underspecified at the moment. There was an analogous issue with the query command: #158 <#158> see the fix: #328 <#328> and the instructions http://robot.obolibrary.org/query Should we have equivalent options for report? However, the defaults may have to be different, because as you point out any ontology that has imports and references classes in the import will produce false positive errors. @jamesaoverton <https://github.com/jamesaoverton> @rctauber <https://github.com/rctauber> how do you feel about having the defaults be different here? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#391 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AP-WRjGb7YXzrg-cZBixy5PDxp9Xi4n_ks5uqM5RgaJpZM4YC9L6> .

cmungall · 2018-10-30T23:12:21Z

I would think that you’re only reporting on your base ontology.

I think what is happening here is that there is leakage of declarations from the import into the base.

E.g. if base has an axiom X SubClassOf R some Y, and Y is in the import, then I believe the declaration that Y is a class leaks into the base, so we end up with a triple in base with Y as subject.

Looking at https://github.com/ontodev/robot/blob/master/robot-core/src/main/resources/report_queries/missing_label.rq#L11

What will then happen is that it will report that Y has no label.

I'm not sure of a good way around this.

That way,
issues with other ontologies won’t break the report.

Yes, this is a problem with my proposal to bring in the import

Not sure of the best way around this

dougli1sqrd · 2018-10-31T01:46:53Z

@cmungall where do you think the Y class get turned into a subject in this? Is this a Jena thing, OWL API thing, or something we do in ROBOT?

I did a small work-around where I also filtered like: FILTER(strstarts(str(?entity), "http://purl.obolibrary.org/obo/GO_") ) and that successfully detects no missing labels. Not sure there is a way to generify that though. It's maybe a data point that the filter works though?

cmungall · 2018-10-31T03:46:11Z

I believe it's OWLAPI, but can you try some SPARQL to check?

beckyjackson · 2018-10-31T03:59:12Z

A declaration with no other axioms is probably an error - should this be a reporting check? It will always come up as missing a label as well, but with this extra check, it may be easier to debug.

…

On Tue, Oct 30, 2018 at 23:46 Chris Mungall ***@***.***> wrote: I believe it's OWLAPI, but can you try some SPARQL to check? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#391 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AP-WRsGQgn1IhF_aUUeDrjuQWZPLxmbsks5uqR0HgaJpZM4YC9L6> .

cmungall · 2018-10-31T04:03:50Z

I believe the OWLAPI injects the triple, I can't recall if this is a consequence of the source file being obo

cmungall · 2018-10-31T04:13:34Z

Yes, annoying

import: http://purl.obolibrary.org/obo/go/imports/cl_import.owl
ontology: test

[Term]
id: TEST:0000001
name: foo
relationship: BFO:0000050 CL:0009004

convert this to any other serialization injects the declaration triple.

But note this isn't completely unique to obo format. If we remove the import (i.e make it a base file) then the declaration is auto-injected.

e.g. if you roundtrip this with robot:

@prefix : <http://purl.obolibrary.org/obo/test.owl#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@base <http://purl.obolibrary.org/obo/test.owl> .

<http://purl.obolibrary.org/obo/test.owl> rdf:type owl:Ontology .


###  http://purl.obolibrary.org/obo/TEST_0000001
<http://purl.obolibrary.org/obo/TEST_0000001> rdf:type owl:Class ;
                                              rdfs:subClassOf [ rdf:type owl:Restriction ;
                                                                owl:onProperty <http://purl.obolibrary.org/obo/BFO_0000050> ;
                                                                owl:someValuesFrom <http://purl.obolibrary.org/obo/CL_0009004>
                                                              ] ;
                                              <http://www.geneontology.org/formats/oboInOwl#id> "TEST:0000001"^^xsd:string ;
                                              rdfs:label "foo"^^xsd:string .

you will see the class and OP declaration injected

jamesaoverton · 2018-10-31T11:44:11Z

Yes, OWLAPI inserts those declarations into serializations.

The SPARQL queries for report are already a bit tricky for reasons like this.

We could add code to strip those bare declarations. Or we could annotate stuff with isDefinedBy #390 and only query for those. There are other options but I can't think of anything elegant.

dosumis · 2018-11-02T15:19:41Z

From discussion in ODK meeting - code could be edited to only check axioms - ignoring declarations.

beckyjackson · 2018-11-02T15:24:40Z

Re @dosumis - the query could also ignore any subjects that only have an rdf:type axiom and no other axioms. This seems like an easy fix.

jamesaoverton · 2018-11-03T12:04:33Z

@rctauber Yes, let's try that approach.

dougli1sqrd · 2018-11-05T17:27:43Z

Last Friday I was trying to write a query like you described, @rctauber, but I was having trouble getting it to work. If you have a way to help, I'd love a pointer!

beckyjackson · 2018-11-05T17:36:54Z

Hi @dougli1sqrd - please look at my PR at #394 (and if you have time, test it 😄) The relevant code is:

  FILTER EXISTS {
    ?entity ?property ?object .
    FILTER (?property != rdf:type)
  }

@ontodev/robot-team I tested this on a few different inputs, but if anybody else has time to test it, I would really appreciate it.

cmungall · 2018-11-05T19:03:32Z

Thanks, we'll test it!

dougli1sqrd · 2018-11-05T20:57:28Z

Ah, cool, so this is working in go-ontology go-edit.obo except for http://purl.obolibrary.org/obo/go# relations. For example:

Level	Rule Name	Subject	Property	Value
ERROR	missing_label	http://purl.obolibrary.org/obo/go#gocheck_do_not_annotate	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#gocheck_do_not_manually_annotate	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_agr	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_aspergillus	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_candida	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_chembl	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_flybase_ribbon	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_generic	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_metagenomics	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_mouse	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_pir	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_plant	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_pombe	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_synapse	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#goslim_yeast	rdfs:label

beckyjackson · 2018-11-05T21:11:31Z

This check is only ignoring entities that only have a declaration triple (x rdf:type y). Looking at these subset defs, these all have a declaration, plus an rdfs:comment annotation and a super-property.

subsetdef: gocheck_do_not_annotate "Term not to be used for direct annotation"

Is translated to:

:gocheck_do_not_annotate rdf:type owl:AnnotationProperty ;
                         rdfs:subPropertyOf oboInOwl:SubsetProperty ;
                         rdfs:comment "Term not to be used for direct annotation"^^xsd:string .

This is an interesting problem because it's unique to ontologies developed in OBO format... @cmungall & @jamesaoverton - do you have any thoughts on this? Potential options I can think of:

ROBOT doesn't worry about handling subset defintions. OBO format ontologies would either need to give their subset definitions a label (easy in OWL, not sure about OBO), or exclude/rewrite this query.
The report queries ignore subproperties of oboInOwl:SubsetProperty. Some ontologies developed in OWL are now annotating the subset definitions with other things though, and may want them to be included in these checks...

dougli1sqrd · 2018-11-05T21:26:44Z

Yeah something like your option 2 could work! Just so you know though there are also a couple of properties in go-edit:

ERROR	missing_label	http://purl.obolibrary.org/obo/go#gocheck_do_not_annotate	rdfs:label		
ERROR	missing_label	http://purl.obolibrary.org/obo/go#gocheck_do_not_manually_annotate	rdfs:label

And would also have to be dealt with somehow. Maybe we could ignore all properties that are subsets of oboInOwl properties? Or something?

cmungall · 2018-11-05T22:08:27Z

1 would be difficult for people

I vote for 2. Either exclude subProps of oboInOwl:SubsetProperty or targets of oboInOwl:inSubset

dougli1sqrd · 2018-11-05T22:10:17Z

Oops I guess I didn't know what I was talking about. My previous comment, gocheck... are actually subsets, so ignore that comment

beckyjackson · 2018-11-05T22:43:01Z

@cmungall - I can exclude the subprops. Should it just be for this label check? I'm guessing they could also be excluded from missing & multiple definitions?

dougli1sqrd · 2018-11-06T01:26:09Z

It looks like this same kind of error is happening for missing_superclass.
Then, I'm not totally sure if this is the same kind of thing as this, but deprecated_class_reference also has a ton of results for go-edit.obo.

ERROR	deprecated_class_reference	GO:0000004	IAO:0000231	IAO:0000227

This is saying GO:0000004 was deprecated because it was merged. And it's merged into GO:0008151 biological process.

This is the owl for GO:0000004:

<owl:Class rdf:about="http://purl.obolibrary.org/obo/GO_0000004">
        <obo:IAO_0000231 rdf:resource="http://purl.obolibrary.org/obo/IAO_0000227"/>
        <obo:IAO_0100001 rdf:resource="http://purl.obolibrary.org/obo/GO_0008150"/>
        <owl:deprecated rdf:datatype="http://www.w3.org/2001/XMLSchema#boolean">true</owl:deprecated>
</owl:Class>

I guess it's catching this because GO:0000004 is deprecated?
Thanks!

cmungall · 2018-11-06T01:33:53Z

This is standard for 50% of ontologies Half make a subclass of ObsoleteClass The other half (including all obo2owl derived) just have the deprecated axiom, with no logical axioms **EDIT** for some reason I thought we were talking about must-have-is-a-parent check. My comment above is irrelevant for labels. For label checks, if ``` <obo:IAO_0000231 rdf:resource="http://purl.obolibrary.org/obo/IAO_0000227"/> ``` is present (which indicates the class was merged), then do exclude from label checks

dosumis · 2018-11-06T10:51:59Z

re subsets: Maybe I'm stating the obvious here - but I don't think we should be checking for labels / definitions of annotation properties.

beckyjackson · 2018-11-06T11:49:50Z

@cmungall & @dougli1sqrd - I can include this line in the deprecated_class_reference check:

FILTER (?property != obo:IAO_0000231) .

That would allow the obsolete classes to use IAO:0000231 as an object property. Obsolete entities are already excluded from the label check.

@dosumis - I think that's reasonable. That would solve a lot of the issues with the default annotation properties, as well. What are others' thoughts on this?

jamesaoverton · 2018-11-06T12:49:56Z

@dosumis Annotation properties should have labels and definitions, like any other term. I don't see why we would exclude them wholesale from these quality checks.

@cmungall Is there some reason you can't add labels and definitions to http://purl.obolibrary.org/obo/go#goslim_flybase_ribbon et al.?

Every one of these "little" exceptions will have to be documented and maintained, and every one of them will make the SPARQL run slower.

cmungall · 2018-11-06T15:39:00Z

unfortunately labels for subsetdefs would get lost in the owl2obo translation. Many ontologies have the edit version maintained in obo for diff/vcs purposes. Subsets have other annoying properties such as hash URIs that don't resolve.

dougli1sqrd · 2018-11-07T00:55:39Z

@rctauber I think your idea would work. I think most/all of those particular false positives are due to merged terms, which will all have that as a ?property (I think).

beckyjackson · 2019-01-30T16:20:31Z

I think this issue has been resolved. We are working on further improvements to the report queries in #429. Please feel free to re-open this issue if any points here need more discussion.

cmungall assigned dougli1sqrd Oct 30, 2018

cmungall changed the title ~~Robot report command seems to merge imports during queries~~ Robot report command does not merge imports during queries, resulting in false positives Oct 30, 2018

dosumis mentioned this issue Nov 2, 2018

No labels check is set to check GO terms by default INCATools/ontology-development-kit#43

Closed

jamesaoverton mentioned this issue Nov 5, 2018

False positives in robot report #323

Closed

jamesaoverton added this to the v1.2.0 milestone Nov 5, 2018

beckyjackson mentioned this issue Nov 5, 2018

Report Query Fixes #394

Merged

beckyjackson mentioned this issue Nov 6, 2018

Report invalid_xrefs check forbids the dash character (-). Should it? #395

Closed

jamesaoverton changed the title ~~Robot report command does not merge imports during queries, resulting in false positives~~ Improvements to report command Nov 6, 2018

jamesaoverton removed this from the v1.2.0 milestone Nov 19, 2018

beckyjackson closed this as completed Jan 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improvements to `report` command #391

Improvements to `report` command #391

dougli1sqrd commented Oct 30, 2018

cmungall commented Oct 30, 2018

beckyjackson commented Oct 30, 2018 via email

cmungall commented Oct 30, 2018

dougli1sqrd commented Oct 31, 2018

cmungall commented Oct 31, 2018

beckyjackson commented Oct 31, 2018 via email

cmungall commented Oct 31, 2018

cmungall commented Oct 31, 2018 •

edited

Loading

jamesaoverton commented Oct 31, 2018

dosumis commented Nov 2, 2018

beckyjackson commented Nov 2, 2018

jamesaoverton commented Nov 3, 2018

dougli1sqrd commented Nov 5, 2018

beckyjackson commented Nov 5, 2018 •

edited

Loading

cmungall commented Nov 5, 2018

dougli1sqrd commented Nov 5, 2018

beckyjackson commented Nov 5, 2018

dougli1sqrd commented Nov 5, 2018

cmungall commented Nov 5, 2018

dougli1sqrd commented Nov 5, 2018

beckyjackson commented Nov 5, 2018

dougli1sqrd commented Nov 6, 2018 •

edited

Loading

cmungall commented Nov 6, 2018 via email •

edited

Loading

dosumis commented Nov 6, 2018

beckyjackson commented Nov 6, 2018

jamesaoverton commented Nov 6, 2018

cmungall commented Nov 6, 2018

dougli1sqrd commented Nov 7, 2018

beckyjackson commented Jan 30, 2019

Improvements to report command #391

Improvements to report command #391

Comments

dougli1sqrd commented Oct 30, 2018

cmungall commented Oct 30, 2018

beckyjackson commented Oct 30, 2018 via email

cmungall commented Oct 30, 2018

dougli1sqrd commented Oct 31, 2018

cmungall commented Oct 31, 2018

beckyjackson commented Oct 31, 2018 via email

cmungall commented Oct 31, 2018

cmungall commented Oct 31, 2018 • edited Loading

jamesaoverton commented Oct 31, 2018

dosumis commented Nov 2, 2018

beckyjackson commented Nov 2, 2018

jamesaoverton commented Nov 3, 2018

dougli1sqrd commented Nov 5, 2018

beckyjackson commented Nov 5, 2018 • edited Loading

cmungall commented Nov 5, 2018

dougli1sqrd commented Nov 5, 2018

beckyjackson commented Nov 5, 2018

dougli1sqrd commented Nov 5, 2018

cmungall commented Nov 5, 2018

dougli1sqrd commented Nov 5, 2018

beckyjackson commented Nov 5, 2018

dougli1sqrd commented Nov 6, 2018 • edited Loading

cmungall commented Nov 6, 2018 via email • edited Loading

dosumis commented Nov 6, 2018

beckyjackson commented Nov 6, 2018

jamesaoverton commented Nov 6, 2018

cmungall commented Nov 6, 2018

dougli1sqrd commented Nov 7, 2018

beckyjackson commented Jan 30, 2019

Improvements to `report` command #391

Improvements to `report` command #391

cmungall commented Oct 31, 2018 •

edited

Loading

beckyjackson commented Nov 5, 2018 •

edited

Loading

dougli1sqrd commented Nov 6, 2018 •

edited

Loading

cmungall commented Nov 6, 2018 via email •

edited

Loading