Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Change RDF handling for querying Projects from DB (DEV-3175) #2989

Merged
merged 56 commits into from
Feb 6, 2024
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
fcb7d53
add first helper methods
BalduinLandolt Jan 9, 2024
027c1b8
use in findOneByQuery
BalduinLandolt Jan 9, 2024
59dcf28
Update KnoraProjectRepoLive.scala
BalduinLandolt Jan 9, 2024
26dea9b
add temporary runner
BalduinLandolt Jan 9, 2024
4539e1a
add simple RDF wrapper
BalduinLandolt Jan 10, 2024
ed4532a
more methods
BalduinLandolt Jan 10, 2024
82d45c2
use it in test code
BalduinLandolt Jan 10, 2024
8325b35
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Jan 16, 2024
0a4ee1a
use ZIO in new RDF model
BalduinLandolt Jan 16, 2024
976623e
fix compilation error
BalduinLandolt Jan 16, 2024
4755e7f
fworking
seakayone Jan 16, 2024
09a3a24
wip
seakayone Jan 16, 2024
9bbd950
add factories to models
BalduinLandolt Jan 16, 2024
597e6fe
continue on rdf model
BalduinLandolt Jan 16, 2024
697e32a
update usage
BalduinLandolt Jan 16, 2024
ff417d0
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Jan 16, 2024
6930f7e
approximate usage to how it could really work
BalduinLandolt Jan 16, 2024
d569fea
add license header
BalduinLandolt Jan 16, 2024
edb4b52
remove parsing from triplestore
BalduinLandolt Jan 16, 2024
169a848
revert changes in RdfFormatUtil
BalduinLandolt Jan 16, 2024
d58a0a1
add factory to new model
BalduinLandolt Jan 16, 2024
754eec4
sort string literals
BalduinLandolt Jan 16, 2024
cc4343b
remove obsolete helper
BalduinLandolt Jan 16, 2024
09b5395
Rename
BalduinLandolt Jan 16, 2024
dbb3b34
fix langString without language
BalduinLandolt Jan 16, 2024
04f8c63
improve sorting
BalduinLandolt Jan 17, 2024
c4d4c67
clean-up
BalduinLandolt Jan 17, 2024
dba7b5b
move query to string
BalduinLandolt Jan 17, 2024
f566f05
extract query to object
BalduinLandolt Jan 17, 2024
4f771b2
remove unused code
BalduinLandolt Jan 17, 2024
a7fa324
move to package
BalduinLandolt Jan 17, 2024
dfd8e3a
rename model
BalduinLandolt Jan 17, 2024
fa9e201
clean-up
BalduinLandolt Jan 17, 2024
cfa69b7
move sorting
BalduinLandolt Jan 17, 2024
8a340a6
move implicits
BalduinLandolt Jan 17, 2024
4d6df64
update TODO
BalduinLandolt Jan 17, 2024
f5388fc
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Jan 22, 2024
b2b3ef4
add some scaladoc
BalduinLandolt Jan 22, 2024
0470d0c
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Feb 5, 2024
b1c5c0c
fix
BalduinLandolt Feb 5, 2024
14e97c0
add documentation
BalduinLandolt Feb 5, 2024
1827091
add tests
BalduinLandolt Feb 5, 2024
8c002f0
streamline the internals
BalduinLandolt Feb 5, 2024
0d3f43d
add streamlining, documentation, tests
BalduinLandolt Feb 5, 2024
6518dce
fmt
BalduinLandolt Feb 5, 2024
336bd10
add missing headers
BalduinLandolt Feb 6, 2024
ddb4ca9
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Feb 6, 2024
944dc1f
continue tests, streamlining, docs
BalduinLandolt Feb 6, 2024
ccd7a9c
add last missing docs and tests
BalduinLandolt Feb 6, 2024
998396e
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Feb 6, 2024
11273b8
remove unnecessary conversions
BalduinLandolt Feb 6, 2024
2b03531
hide jena implementation details
BalduinLandolt Feb 6, 2024
b5305dc
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Feb 6, 2024
c6951b3
manage resources in stringreader
BalduinLandolt Feb 6, 2024
2e29b1c
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Feb 6, 2024
db90508
Merge branch 'main' into wip/improve-rdf-handling
BalduinLandolt Feb 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/*
* Copyright © 2021 - 2024 Swiss National Data and Service Center for the Humanities and/or DaSCH Service Platform contributors.
* SPDX-License-Identifier: Apache-2.0
*/

package org.knora.webapi.slice.admin.repo.rdf

import dsp.valueobjects.V2
import org.knora.webapi.slice.admin.domain.model.KnoraProject.*
import org.knora.webapi.slice.common.repo.rdf.LangString

object RdfConversions {
implicit val shortcodeConverter: String => Either[String, Shortcode] =
Shortcode.from

implicit val shortnameConverter: String => Either[String, Shortname] =
Shortname.from

implicit val longnameConverter: String => Either[String, Longname] =
Longname.from

implicit val descriptionConverter: LangString => Either[String, Description] = langString =>
Description.from(V2.StringLiteralV2(langString.value, langString.lang))

implicit val keywordConverter: String => Either[String, Keyword] =
Keyword.from

implicit val logoConverter: String => Either[String, Logo] =
Logo.from

implicit val statusConverter: Boolean => Either[String, Status] =
value => Right(Status.from(value))

implicit val selfjoinConverter: Boolean => Either[String, SelfJoin] =
value => Right(SelfJoin.from(value))
}
mpro7 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -23,37 +23,97 @@ import org.knora.webapi.slice.admin.domain.model.KnoraProject.*
import org.knora.webapi.slice.admin.domain.model.RestrictedView
import org.knora.webapi.slice.admin.domain.model.RestrictedViewSize
import org.knora.webapi.slice.admin.domain.service.KnoraProjectRepo
import org.knora.webapi.slice.admin.repo.rdf.RdfConversions.*
import org.knora.webapi.slice.admin.repo.service.KnoraProjectQueries.getProjectByIri
import org.knora.webapi.slice.common.repo.rdf.Errors.RdfError
import org.knora.webapi.slice.common.repo.rdf.RdfModel
import org.knora.webapi.slice.common.repo.service.PredicateObjectMapper
import org.knora.webapi.slice.resourceinfo.domain.InternalIri
import org.knora.webapi.store.triplestore.api.TriplestoreService
import org.knora.webapi.store.triplestore.api.TriplestoreService.Queries.Construct
import org.knora.webapi.store.triplestore.api.TriplestoreService.Queries.Update

object KnoraProjectQueries {
private[service] def getProjectByIri(iri: ProjectIri): Construct =
Construct(
s"""|PREFIX knora-admin: <http://www.knora.org/ontology/knora-admin#>
|PREFIX knora-base: <http://www.knora.org/ontology/knora-base#>
|PREFIX owl: <http://www.w3.org/2002/07/owl#>
|CONSTRUCT {
| ?project ?p ?o .
| ?project knora-admin:belongsToOntology ?ontology .
|} WHERE {
| BIND(IRI("${iri.value}") as ?project)
| ?project a knora-admin:knoraProject .
| OPTIONAL {
| ?ontology a owl:Ontology .
| ?ontology knora-base:attachedToProject ?project .
| }
| ?project ?p ?o .
|}""".stripMargin
)
}

final case class KnoraProjectRepoLive(
private val triplestore: TriplestoreService,
private val mapper: PredicateObjectMapper,
private implicit val sf: StringFormatter
) extends KnoraProjectRepo {

override def findById(id: ProjectIri): Task[Option[KnoraProject]] =
findOneByQuery(sparql.admin.txt.getProjects(maybeIri = Some(id.value), None, None))
private val belongsToOntology = "http://www.knora.org/ontology/knora-admin#belongsToOntology"

override def findById(id: ProjectIdentifierADM): Task[Option[KnoraProject]] = {
val maybeIri = id.asIriIdentifierOption
val maybeShortname = id.asShortnameIdentifierOption
val maybeShortcode = id.asShortcodeIdentifierOption
findOneByQuery(
sparql.admin.txt
.getProjects(maybeIri = maybeIri, maybeShortname = maybeShortname, maybeShortcode = maybeShortcode)
)
}
override def findById(id: ProjectIri): Task[Option[KnoraProject]] = findOneByIri(id)

override def findById(id: ProjectIdentifierADM): Task[Option[KnoraProject]] =
id.asIriIdentifierOption match {
case Some(iri) => findOneByIri(ProjectIri.unsafeFrom(iri))
case None =>
val maybeShortname = id.asShortnameIdentifierOption
val maybeShortcode = id.asShortcodeIdentifierOption
findOneByQuery(
sparql.admin.txt
.getProjects(None, maybeShortname = maybeShortname, maybeShortcode = maybeShortcode)
)
}

private def findOneByIri(iri: ProjectIri): Task[Option[KnoraProject]] =
for {
ttl <- triplestore.queryRdf(getProjectByIri(iri))
newModel <- RdfModel.fromTurtle(ttl)
project <- toKnoraProjectNew(newModel, iri).option
} yield project

private def findOneByQuery(query: TxtFormat.Appendable): Task[Option[KnoraProject]] =
for {
construct <- triplestore.query(Construct(query)).flatMap(_.asExtended).map(_.statements.headOption)
project <- ZIO.foreach(construct)(toKnoraProject)
} yield project

private def toKnoraProjectNew(model: RdfModel, iri: ProjectIri): IO[RdfError, KnoraProject] =
for {
resource <- model.getResource(iri.value)
shortcode <- resource.getStringLiteralOrFail[Shortcode](ProjectShortcode)
shortname <- resource.getStringLiteralOrFail[Shortname](ProjectShortname)
longname <- resource.getStringLiteral[Longname](ProjectLongname)
description <- resource.getLangStringLiteralsOrFail[Description](ProjectDescription)
keywords <- resource.getStringLiterals[Keyword](ProjectKeyword)
logo <- resource.getStringLiteral[Logo](ProjectLogo)
status <- resource.getBooleanLiteralOrFail[Status](StatusProp)
selfjoin <- resource.getBooleanLiteralOrFail[SelfJoin](HasSelfJoinEnabled)
ontologies <- resource.getObjectIris(belongsToOntology)
} yield KnoraProject(
id = iri,
shortcode = shortcode,
shortname = shortname,
longname = longname,
description = description,
keywords = keywords.toList.sortBy(_.value),
logo = logo,
status = status,
selfjoin = selfjoin,
ontologies = ontologies.toList
)

override def findAll(): Task[List[KnoraProject]] = {
val query = sparql.admin.txt.getProjects(None, None, None)
for {
Expand Down