Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rdf output adapter #338

Merged
merged 29 commits into from
Apr 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
f426463
First minimal working version for rdf conversion
supermaxiste Jun 22, 2023
f1f57de
Bump pillow from 10.0.1 to 10.3.0
dependabot[bot] Apr 4, 2024
9a6596e
Merge pull request #1 from thehyve/dependabot/pip/pillow-10.3.0
LoesvdBiggelaar Apr 4, 2024
0e06071
Merge remote-tracking branch 'origin/main' into rdf
Apr 4, 2024
94a7207
update the RDF writer to be compatible with latest changes of Biocypher.
LoesvdBiggelaar Apr 5, 2024
3738925
add triples with URIs instead of strings. also add properties of the …
LoesvdBiggelaar Apr 8, 2024
5daec20
include some namespaces. Currently it's hardcoded but it should be in…
LoesvdBiggelaar Apr 9, 2024
ab4905e
write files for every node and relationship type , instead of one big…
LoesvdBiggelaar Apr 11, 2024
301927f
remove redundant code from rdfwriter which is already defined in batc…
LoesvdBiggelaar Apr 11, 2024
e93f1ee
add edges as nodes. attach the properties of the edge to the created …
LoesvdBiggelaar Apr 17, 2024
6dd5590
create basic unit test. add dummy nodes and edges and check nr of nod…
LoesvdBiggelaar Apr 17, 2024
588bb04
add namespaces to the rdf output
LoesvdBiggelaar Apr 17, 2024
7656700
Merge remote-tracking branch 'origin/main' into rdf
LoesvdBiggelaar Apr 18, 2024
1a6c3fb
resolve merge conflict
LoesvdBiggelaar Apr 18, 2024
9853801
use example data from fixtures and assert every node and 'edge' has a…
LoesvdBiggelaar Apr 18, 2024
75661be
run precommit, align with pep8 naming convention
LoesvdBiggelaar Apr 18, 2024
21bb202
#106: fix integration test
nilskre Apr 18, 2024
74a951d
add some doc
LoesvdBiggelaar Apr 18, 2024
f5d85b9
small changes
LoesvdBiggelaar Apr 18, 2024
c58d64f
Merge pull request #337 from thehyve/rdf
slobentanzer Apr 18, 2024
337f1a4
#106: minor fixes; link rdf docs from main output docs page
nilskre Apr 19, 2024
9c18792
expand tests to check every triple individually. needed to rearrange/…
LoesvdBiggelaar Apr 22, 2024
51ba020
docs format and phrasing
slobentanzer Apr 24, 2024
fc14271
move output in TOC
slobentanzer Apr 24, 2024
2d47b6a
output page index formatting
slobentanzer Apr 24, 2024
4b3860c
pre-commit
slobentanzer Apr 24, 2024
861c9d0
bump action version due to deprecation warning
slobentanzer Apr 24, 2024
e38bf3b
move python versions to 3.10-3.12 due to
slobentanzer Apr 24, 2024
802f18f
#342: temporary cicd fix (set macos version to last macos version on …
nilskre Apr 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .github/actions/setup/action.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
name: 'Setup'
description: 'Setup Python and Poetry'
name: "Setup"
description: "Setup Python and Poetry"
inputs:
PYTHON_VERSION:
description: 'Python version'
description: "Python version"
default: 3.9
POETRY_VERSION:
description: 'Poetry version'
description: "Poetry version"
default: 1.5.1

runs:
using: "composite"
steps:
- name: Set up Python ${{inputs.PYTHON_VERSION}}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{inputs.PYTHON_VERSION}}
- name: Load cached Poetry installation
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ jobs:
strategy:
fail-fast: true
matrix:
os: ["ubuntu-latest", "macos-latest", "windows-latest"]
python-version: ["3.9", "3.10", "3.11"]
os: ["ubuntu-latest", "macos-13", "windows-latest"]
python-version: ["3.10", "3.11", "3.12"]
neo4j-version: ["4.4-enterprise", "5.17.0-enterprise"]
exclude:
# Windows tests don't use the PostgreSQL/Neo4j Docker containers
Expand Down
4 changes: 4 additions & 0 deletions biocypher/_config/biocypher_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,10 @@ postgresql:
# import_call_bin_prefix: '' # path to "psql"
# import_call_file_prefix: '/path/to/files'

rdf:
### RDF configuration ###
rdf_format: turtle

sqlite:
### SQLite configuration ###

Expand Down
11 changes: 11 additions & 0 deletions biocypher/write/_batch_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import re
import glob

from rdflib import Graph
from more_itertools import peekable

from biocypher._create import BioCypherEdge, BioCypherNode, BioCypherRelAsNode
Expand Down Expand Up @@ -117,6 +118,8 @@ def __init__(
db_password: str = None,
db_host: str = None,
db_port: str = None,
rdf_format: str = None,
rdf_namespaces: dict = {},
):
"""

Expand Down Expand Up @@ -196,12 +199,20 @@ class contains all methods expected by a bach writer instance, some of

db_port:
The database port.

rdf_format:
The format of RDF.

rdf_namespaces:
The namespaces for RDF.
"""
self.db_name = db_name
self.db_user = db_user
self.db_password = db_password
self.db_host = db_host or "localhost"
self.db_port = db_port
self.rdf_format = rdf_format
self.rdf_namespaces = rdf_namespaces

self.delim, self.escaped_delim = self._process_delimiter(delimiter)
self.adelim, self.escaped_adelim = self._process_delimiter(
Expand Down
5 changes: 5 additions & 0 deletions biocypher/write/_write.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"""

from biocypher._logger import logger
from biocypher.write.graph._rdf import _RDFWriter
from biocypher.write.graph._neo4j import _Neo4jBatchWriter
from biocypher.write.graph._arangodb import _ArangoDBBatchWriter
from biocypher.write.relational._sqlite import _SQLiteBatchWriter
Expand Down Expand Up @@ -43,6 +44,8 @@
"ArangoDB": _ArangoDBBatchWriter,
"sqlite": _SQLiteBatchWriter,
"sqlite3": _SQLiteBatchWriter,
"rdf": _RDFWriter,
"RDF": _RDFWriter,
}


Expand Down Expand Up @@ -102,4 +105,6 @@ def get_writer(
db_user=dbms_config.get("user"), # psql
db_password=dbms_config.get("password"), # psql
db_port=dbms_config.get("port"), # psql
rdf_format=dbms_config.get("rdf_format"), # rdf
rdf_namespaces=dbms_config.get("rdf_namespaces"), # rdf
)
Loading
Loading