Quality Issues
cmader edited this page May 15, 2012
·
33 revisions
The vocabulary quality issues defined in the following sections should be applicable to any SKOS vocabulary. Some of the issues are taken from already existing research (see section "Related Work"). Others reflect our thoughts when investigating real-world thesauri (see the data repository). If not stated otherwise, we treat the SKOS vocabularies as fully entailed RDFS graphs. We also enrich the vocabularies by entailment of owl:inverseOf properties as well as instances of owl:TransitiveProperty and owl:SymmetricProperty.
| Description | To make the vocabulary more convenient for humans to use, instances of SKOS classes (Concept, ConceptScheme, Collection) should be labeled using e.g., prefLabel, altLabel, rdfs:label, dc:title. |
| Example | |
| Preliminary ideas on computation | Not implemented yet. Create a SPARQL query, finding all statements involving (subproperties of) rdfs:label for all authoritative concepts. Those concepts having no such statements are returned. |
| Description | Some controlled vocabularies contain literals in natural language, but without information what language has actually been used. Language tags might also not conform to language standards, such as RFC 3066. |
| Example | The network ontology defines a prefLabel of the resource http://river.styx.org/network#AddressFamilies that has no language information. Furthermore, the press contacts information dataset (http://data.southampton.ac.uk/dataset/pressinfo.html) defines labels for resources without stating the language. |
| Implementation | Iteration over all triples in the vocabulary that have a predicate which is a (subclass of) rdfs:label or skos:note. |
| Description | Some concepts in a thesaurus are labeled in only one language, some in multiple languages. It may be desirable to have each concept labeled in each of the languages that also are used on the other concepts. So a quality issue might be the incompleteness of language coverage throughout all concepts. |
| Example | The MARC List for Countries (http://id.loc.gov/vocabulary/countries.html) defines french labels for some but not all labels. |
| Implementation | Iteration over all concepts in the vocabulary and creation of a global set of language tags appearing in the vocabulary. In a second iteration, each concept having a set of language tags that is not equal to the global language tag set is returned. |
| Description | According to the SKOS-Reference, section 5.4 (Integrity Conditions) prefLabel, altLabel and hiddenLabel are pairwise disjoint properties. However, there are numerous vocabularies violating this definition. |
| Example | http://www.afp-ifm-thesaurus.net/t-pro/Merkzeichen prefLabel and altLabel identical (marques@fr) http://lod.geospecies.org/kingdoms/Af?format=rdf prefLabel and altLabel identical (Bacteria) http://www.afp-ifm-thesaurus.net/t-pro/Motto prefLabel and hiddenLabel identical (motto@de) |
| Implementation | Iteration over all triples in the vocabulary and filter for subjects that have identical values for their skos:prefLabel, skos:altLabel and skos:hiddenLabel triples. |
| Description | The SKOS "standard" defines a number of properties useful for documenting the meaning of the concepts in a thesaurus (section 7) also in a human-readable form. Intense use of these properties leads to a well-documented thesaurus which should also improve its quality. |
| Example | Library of Congress Thesaurus for Graphic Materials offers a high coverage of documentation properties |
| Implementation | Iteration over all concepts in the vocabulary and find those not using one of skos:note, skos:changeNote, skos:definition, skos:editorialNote, skos:example, skos:historyNote, or skos:scopeNote |
| Description | This is a generalization of a recommendation in the SKOS primer, that “no two concepts have the same preferred lexical label in a given language when they belong to the same concept scheme”. This could indicate missing disambiguation information and thus lead to problems in autocompletion application. |
| Example | In the concepts http://purl.org/collections/nl/am/t-8523 a skos:prefLabel is defined as "streekvervoer" ("local transport"). Another concept, http://purl.org/collections/nl/am/t-6081 defines skos:altLabel "streekvervoer" and skos:prefLabel "openbaar vervoer" ("public transport"). |
| Implementation | Iteration over all authoritative concepts, collecting their respective labels. In a second pass, similarity of all possible label pairs is checkt by a similarity function. Concept labels with a similarity value below a given threshold, are considered conflicting and are returned. In the current implementation, the similarity function is string equality with a threshold equal to 1. |
SKOS is based on RDF, which is a graph-based data model. Therefore we can concentrate on the vocabulary's graph-based structure for assessing the quality of SKOS vocabularies and apply graph- and network-analysis techniques.
| Description | An orphan concept is a concept without any associative or hierarchical relations. It might have attached literals like e.g., labels, but is not connected to any other resource, lacking valuable context information. A controlled vocabulary that contains many orphan concepts is less usable for search and retrieval use cases, because, e.g., no hierarchical query expansion can be performed on search terms to find documents with more general content. |
| Example | In the press contacts information dataset from the University of Southampton (http://data.southampton.ac.uk/dataset/pressinfo.html), SKOS concepts are defined but aren't linked to other resources using SKOS properties (e.g., http://id.southampton.ac.uk/pressinfo/subject/ActiveSourceSeismology). Similarly, the http://river.styx.org/network ontology defines the concept http://river.styx.org/network#AddressFamily, but doesn't link it using SKOS properties. |
| Implementation | Iteration over all concepts in the vocabulary and returning that don't have associated resources using (subproperties of) skos:semanticRelation. |
| Description | Checking the connectivity of the graph, it is possible to identify all weakly connected components. These datasets form "islands" in the vocabulary and might be caused by incomplete data acquisition, "forgotten" test data, outdated terms and the like. |
| Example | The dmGeo vocabulary consists of 5 weakly connected components. It was available at http://www.dismarc.org but now seems to be offline. Weakly connected components can also be found in the LVAk thesaurus. |
| Implementation | Creation of an undirected graph that includes all non-orphan concepts as nodes and all semantic relations as edges. Tarjan's algorithm then finds and returns all weakly connected components. |
| Description | Although perfectly consistent with the SKOS data model, cyclic relations may reveal a logical problem in the thesaurus. Consider the following example: "decision" -> "problem resolution" -> "problem" (-> "decision": here the cycle is closed). The concepts are connected using skos:broader relationships (indicated with "->"). Due to the fact that a thesaurus is in many cases a product of consensus between the contributors (or just the decision of one dedicated thesaurus manager), it will be almost impossible to automatically resolve the cycle (i.e. deleting an edge). |
| Example | dbpedia categories: http://dbpedia.org/resource/Category:Wikipedians_in_West_Virginia is related broader to itself. http://dbpedia.org/resource/Category:Republic_of_Macedonia broader http://dbpedia.org/resource/Category:Macedonia broader http://dbpedia.org/resource/Category:Geography_of_the_Republic_of_Macedonia furthermore: http://dbpedia.org/resource/Category:Graphics_software broader http://dbpedia.org/resource/Category:Application_software and vice versa |
| Implementation | Construction of a graph having all concepts as nodes and the set of edges being skos:broader relations. |
| Description | Two concepts are sibling, but also connected by an associative relation. In that context, the associative relation is not necessary. See ISO_DIS_25964-1, 11.3.2.2 |
| Example | |
| Implementation | Identification of all pairs of concepts that have the same broader or narrower concepts, i.e. they are "sibling terms". All siblings that are related by a skos:related property are returned. |
| Description | skos:broaderTransitive and skos:narrowerTransitive are, according to the SKOS reference document, "not used to make assertions", so they should not be the only relations hierarchically relating two concepts. |
| Example | |
| Implementation | Identification of all concept pairs that are related by skos:broaderTransitive or skos:narrowerTransitive properties but not by their skos:broader and skos:narrower subproperties. |
| Description | Reciprocal relations (e.g., broader/narrower, related) should be included in the controlled vocabularies to, e.g., to achieve better search results using SPARQL in systems without reasoner support. |
| Example | |
| Implementation | This issue is checked WITHOUT inference of owl:inverseOf properties. We iterate over all triples and check for each property if an inverse property is defined in the SKOS ontology and if the respective statement using this property is included in the vocabulary. If not, the resources associated with this property are returned. |
| Description | A vocabulary should provide "entry points" to the data to provide “efficient access” (SKOS primer) and guidance for human users. |
| Example | |
| Implementation | For every ConceptScheme in the controlled vocabulary, a SPARQL query is issued finding resources that are associated with this ConceptScheme by one of the properties skos:hasTopConcept or skos:topConceptOf. |
| Description | Concepts "internal to the tree" should not be indicated as top concepts, as pointed out in [Allemang2011]. |
| Example | |
| Implementation | A SPARQL query finds all top concepts (being defined by one of the properties skos:hasTopConcept or skos:topConceptOf) having associated a broader concept. |
When publishing Linked Data, it is important to respect the following "rules" (also see http://www.w3.org/DesignIssues/LinkedData.html):
- data is provided using standard formats (e.g., RDF which is obviously the case for SKOS vocabularies)
- linked resources are dereferencable and provide further information
- data linked to and from other resources
| Description | The usage of its concepts can be an indicator for a vocabulary's quality. Usage can be determined by the number of external resources, referencing these concepts. |
| Example |
Consider the concept "http://dbpedia.org/resource/Michael_Jackson". It is, for example, referenced by the following resources:
|
| Implementation | For each authoritative concept in the vocabulary, a SPARQL query (against, e.g. the Sindice endpoint) is issued that returns all triples in which the concept shows up as the object. An estimation of the number of other vocabularies referencing the concept can be obtained by examining if the host part of the returned triple subject URIs does't match the publishing host of the vocabulary. Concepts for which no such matches can be found are returned. |
| Description |
SKOS concepts can define links to other concepts within one and the same vocabulary, to concepts in other vocabularies, or to external resources on the Web. These external links are essential to, for example,
|
| Example | The New York Times People Vocabulary is aligned to dbpedia using owl:sameAs properties (e.g., http://data.nytimes.com/64870337666324078863). |
| Implementation | For each authoritative concept in the vocabulary, a SPARQL query is issued that returns all IRIs that occur as subject or object in triples where this concept is involved. All IRIs that are HTTP URIs and refer to a non-authoritative resource for the concept are counted. Concepts with a count that equals zero are returned. |
| Description | If concepts link to other resources (link targets) on the Web, it is important that these resources are dereferencable and return a response code other than 200 after possible redirections. |
| Example | New York Times People Vocabulary: Response 404 when dereferencing http://data.nytimes.com/elements/manual |
| Implementation | A SPARQL query is issued that finds all HTTP URIs being part (as subject, predicate, or object) of a triple in the vocabulary. The found URIs are then dereferenced and returned if the HTTP response code is other than 200. |
| Description | The vocabulary should not invent any new terms within the SKOS namespace or use “deprecated” SKOS elements like those defined in Appendix D of the SKOS reference. |
| Example | NSDL Registry Agents Vocabulary uses skos:status, which is not defined in http://www.w3.org/2009/08/skos-reference/skos.rdf. http://vocabularyserver.com/emergencias/?tema=575 uses skos:subjectIndicator |
| Implementation | A SPARQL query finds all IRIs that appear in one of the vocabulary's triples in combination with a "deprecated" predicate. "Invented" new terms are found by a SPARQL query, selecting all IRIs in the vocabulary's RDF triples belonging to the SKOS namespace but are not defined in the SKOS ontology. All terms found by the two mentioned queries are returned. |
| Description | URIs should be dereferencable. C. Bizer, How to Publish Linked Data on the Web: "In the context of Linked Data, we restrict ourselves to using HTTP URIs only and avoid other URI schemes such as URNs and DOIs." |
| Example | In CFR Thesaurus (Thesaurus in the Legal domain by the Cornell University) a concept has been identified by a file:// URI |
| Implementation | A SPARQL query is used to find all IRIs that occur as subject in the vocabulary's RDF triples. If their protocol identifier is other than http or https, the resource is returned. |
This category defines issues that relate to specific design decisions of the SKOS ontology. Some of them are also semi-formally expressed in the SKOS reference documentation.
| Description | Covers condition S27 from the SKOS reference document, that has not been defined formally. |
| Example | |
| Implementation | In a first step, all pairs of concepts are found that are associatively connected, using a SPARQL query. In the second step, a graph is created, containing only hierarchically related concepts and the respective relations. For each concept pair from the first step, we check for a path in the graph from step two. If such a path is found, a clash has been identified and the causing concepts are returned. |
| Description | Covers condition S46 from the SKOS reference document, that has not been defined formally. |
| Example | |
| Implementation | Can be solved by issuing a SPARQL query. |
| Description | According to the SKOS reference document, "A resource has no more than one value of skos:prefLabel per language tag". |
| Example | |
| Implementation | A SPARQL query is used to find concepts with at least two prefLabels. In a second step, the language tags of these prefLabels are analyzed and an ambiguity is detected if they are equal. |
| Description | Covers condition S13 from the SKOS reference document, stating that "skos:prefLabel, skos:altLabel and skos:hiddenLabel are pairwise disjoint properties". |
| Example | |
| Implementation | A SPARQL query collects all labels of all concepts, building an in-memory structure. This structure is then checked for disjoint entries. |