Skip to content

Commit

Permalink
adds CSV serialisations for each RDF graph
Browse files Browse the repository at this point in the history
- adds a <file>.csv for each RDF graph present e.g. both /dpv/dpv.csv
  and /dpv/modules/<module>.csv are provided
- see #165 for discussion
- contents of CSV are: 'term', 'type', 'iri', 'label', 'definition',
  'dpvtype', 'subclassof', 'hasbroader', 'scopenote', 'created', 'modified',
  'vocab', 'namespace'
- where:
  - term is the DPV term without vocab namespace e.g. AcademicResearch
  - type represents if it is a rdfs:Class or rdf:Property
  - dpvtype is the DPV term whose instance the term is e.g. Purpose
  - subclassof indicates the DPV term that is used with rdfs:subClasOf
  - hasbroader lists the DPV terms used with skos:broader
  - vocab provides the vocab identifier e.g. dpv
  - namespace provides the vocab IRI e.g. https://w3id.org/dpv
  - iri = namespace + term
  • Loading branch information
coolharsh55 committed Jun 16, 2024
1 parent fcc764c commit 9890493
Show file tree
Hide file tree
Showing 111 changed files with 6,316 additions and 1 deletion.
8 changes: 8 additions & 0 deletions ai/ai.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
term,type,iri,label,definition,dpvtype,subclassof,hasbroader,scopenote,created,modified,vocab,namespace
AI,class,https://w3id.org/dpv/ai#AI,Artificial Intelligence (AI),"A technical and scientific field devoted to the engineered system that generates outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives",,https://w3id.org/dpv#Technology,https://w3id.org/dpv#Technology,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
AISystem,class,https://w3id.org/dpv/ai#AISystem,AI System,"OECD 2024 definition: An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment. ISO/IEC 22989:2023 definition: engineered system that generates outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives",,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,,2024-05-17,,ai,https://w3id.org/dpv/ai
Capability,class,https://w3id.org/dpv/ai#Capability,Capability,Capability or use of AI to achieve a technical goal or objective,,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
Measure,class,https://w3id.org/dpv/ai#Measure,Measure,Measure to address risk associated with AI Systems,,https://w3id.org/dpv#RiskMitigationMeasure,https://w3id.org/dpv#RiskMitigationMeasure,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
Model,class,https://w3id.org/dpv/ai#Model,Model,"Physical, mathematical or otherwise logical representation of a system, entity, phenomenon, process or data",,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,,2024-05-17,,ai,https://w3id.org/dpv/ai
Risk,class,https://w3id.org/dpv/ai#Risk,Risk,"Risk associated with development, use, or operation of AI systems",,https://w3id.org/dpv#Risk,https://w3id.org/dpv#Risk,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
Technique,class,https://w3id.org/dpv/ai#Technique,Technique,Techniques for using or applying AI,,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
2 changes: 2 additions & 0 deletions ai/modules/capabilities.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
term,type,iri,label,definition,dpvtype,subclassof,hasbroader,scopenote,created,modified,vocab,namespace
Capability,class,https://w3id.org/dpv/ai#Capability,Capability,Capability or use of AI to achieve a technical goal or objective,,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
4 changes: 4 additions & 0 deletions ai/modules/core.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
term,type,iri,label,definition,dpvtype,subclassof,hasbroader,scopenote,created,modified,vocab,namespace
AI,class,https://w3id.org/dpv/ai#AI,Artificial Intelligence (AI),"A technical and scientific field devoted to the engineered system that generates outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives",,https://w3id.org/dpv#Technology,https://w3id.org/dpv#Technology,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
AISystem,class,https://w3id.org/dpv/ai#AISystem,AI System,"OECD 2024 definition: An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment. ISO/IEC 22989:2023 definition: engineered system that generates outputs such as content, forecasts, recommendations or decisions for a given set of human-defined objectives",,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,,2024-05-17,,ai,https://w3id.org/dpv/ai
Model,class,https://w3id.org/dpv/ai#Model,Model,"Physical, mathematical or otherwise logical representation of a system, entity, phenomenon, process or data",,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,,2024-05-17,,ai,https://w3id.org/dpv/ai
2 changes: 2 additions & 0 deletions ai/modules/measures.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
term,type,iri,label,definition,dpvtype,subclassof,hasbroader,scopenote,created,modified,vocab,namespace
Measure,class,https://w3id.org/dpv/ai#Measure,Measure,Measure to address risk associated with AI Systems,,https://w3id.org/dpv#RiskMitigationMeasure,https://w3id.org/dpv#RiskMitigationMeasure,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
2 changes: 2 additions & 0 deletions ai/modules/risks.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
term,type,iri,label,definition,dpvtype,subclassof,hasbroader,scopenote,created,modified,vocab,namespace
Risk,class,https://w3id.org/dpv/ai#Risk,Risk,"Risk associated with development, use, or operation of AI systems",,https://w3id.org/dpv#Risk,https://w3id.org/dpv#Risk,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
2 changes: 2 additions & 0 deletions ai/modules/techniques.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
term,type,iri,label,definition,dpvtype,subclassof,hasbroader,scopenote,created,modified,vocab,namespace
Technique,class,https://w3id.org/dpv/ai#Technique,Technique,Techniques for using or applying AI,,https://w3id.org/dpv/ai#AI,https://w3id.org/dpv/ai#AI,This concept is a stub;This concept is a stub,2024-04-28,,ai,https://w3id.org/dpv/ai
60 changes: 60 additions & 0 deletions code/200_serialise_RDF.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,64 @@ def _convert(term:dict) -> dict:

# == export ==

def write_CSV_graph(graph, filepath: str, vocab: str, namespace: str) -> None:
'''Write a CSV file for the given graph at filepath'''
def _consolidate(item: list) -> list:
items = set()
for item in item.split(';'):
if not item.startswith('https://w3id.org/dpv'): continue
items.add(item)
return ";".join(items)

query_header = ('iri', 'label', 'definition',
'dpvtype', 'subclassof', 'hasbroader',
'scopenote', 'created', 'modified')
header = ('term', 'type', *query_header, 'vocab', 'namespace')
with open(f'{filepath}.csv', 'w') as fd:
writer = csv.writer(fd)
writer.writerow(header)
query = """
SELECT ?iri ?label ?definition
(group_concat(?type; separator=";") as ?types)
(group_concat(?parent; separator=";") as ?subclassof)
(group_concat(?broader; separator=";") as ?hasbroader)
(group_concat(?note; separator=";") as ?scopenote)
?created ?modified
WHERE {
?iri a skos:Concept .
?iri skos:prefLabel ?label .
?iri rdf:type ?type .
?iri skos:definition ?definition .
?iri dct:created ?created .
OPTIONAL { ?iri rdfs:subClassOf ?parent } .
OPTIONAL { ?iri skos:broader ?broader } .
OPTIONAL { ?iri skos:scopeNote ?note } .
OPTIONAL { ?iri dct:modified ?modified } .
} GROUP BY ?iri ?label ?definition ORDER BY ?iri
"""
results = graph.query(query)
for row in results:
value = {
'term': prefix_from_iri(row[0]).split(':')[1],
'vocab': vocab,
'namespace': namespace}
for index, item in enumerate(row):
value[query_header[index]] = item
if 'rdf-schema#Class' in value['dpvtype']:
value['type'] = 'class'
elif 'rdf-syntax-ns#Property' in value['dpvtype']:
value['type'] = 'property'
else:
raise Exception(f"Unknown type: {value['dpvtype']} in {value['term']}")
for key in (('dpvtype', 'subclassof', 'hasbroader')):
value[key] = _consolidate(value[key])

writer.writerow((value[x] for x in header))


INFO(f'wrote {filepath}.csv')


# === serialise-RDF ===
def serialize_graph(triples:list, filepath:str, vocab:str, hook:str=None) -> None:
'''`serialize_graph` serializes triples at filepath with defined
Expand Down Expand Up @@ -251,6 +309,8 @@ def serialize_graph(triples:list, filepath:str, vocab:str, hook:str=None) -> Non
graph.serialize(f'{filepath}.{ext}', format=format)
INFO(f'wrote {filepath}.[{",".join(RDF_SERIALIZATIONS)}]')

write_CSV_graph(graph, filepath, vocab, vocab_iri)

##### Serialise in OWL
# - TODO: Decide format for serialising OWL variant
# - TODO: convert to manchester syntax via external tool
Expand Down
Loading

0 comments on commit 9890493

Please sign in to comment.