Skip to content
Permalink
Browse files
feat!: add archive representation to DSP-API (DEV-17) (#1926)
* feat: add bundle representation to knora-base ontology and constants

* fix: overlook in knora-base ontology

* test: try to break stuff that should break

* feat: un-break tests as good as possible

* refactor: move some json-ld generation to model

* refactor: tidy up a bit

* refactor: add ChangeFileRequest to FileModels

* chore: update test data according to new knora-base ontology

* refactor: get rid of some code smells

* refactor: remove code smell

* refactor: move more json-ld generation into FileModels

* temporarily disable all sipi integration tests

* fix: error in books ontology

* bring back sipi inntegration test

* ignore failing tests

* test: move more json-ld generation to FileModels

* refactor: further simplify generating json-ld through FileModels

* refactor: improve file model

* feat: add bundle to lua scripts

* test: temporarily ignore some tests

* feat: forbid bundles in document file type

* test: add test for uploading a BundleRepresentation

* test: add test for changing BundleRepresentation

* refactor: improve file models

* docs: update documentation

* docs: update documentation

* docs: add documentation and auto-reformat some markdown

* test: add unit test for creating a DocumentRepresentation

* refactor: start using models for unit tests

* refactor: generalize file message model

* refactor: further generalize file message model

* refactor: expand file message models

* test: add unit test for creating a bundle representation

* refactor: tidy up books ontology

* docs: update documentation

* refactor: move some string literals into constants

* docs: add scaladoc comments to FileModels

* refactor: final clean up

* docs: update docs according to review

* fix: remove wrong properties from ArchiveRepresentation

* refactor: rename bundle to archive

* test: update test data

* refactor: change file models

* refactor: change file models

* refactor: rework file models

* fix: oversight in json-ld generation in file models

* fix: use wrong IRI in FileModels toMessage function

* fix: expect correct permission for archive representation

* test: add test for file models

* test: add more file model tests

* test: improve file model tests
  • Loading branch information
BalduinLandolt committed Nov 25, 2021
1 parent fe6106f commit 0123a8f62b08c31e053c337d22c9d8dffaf321c5
Showing with 8,275 additions and 6,466 deletions.
  1. +517 −855 docs/02-knora-ontologies/knora-base.md
  2. +5 −0 docs/03-apis/api-v2/editing-values.md
  3. +66 −19 knora-ontologies/knora-base.ttl
  4. +27 −7 sipi/scripts/file_info.lua
  5. +2 −2 test_data/ontologies/books-onto.ttl
  6. +452 −452 test_data/ontologyR2RV2/anythingOntologyWithValueObjects.ttl
  7. +76 −76 test_data/ontologyR2RV2/anythingThingWithAllLanguages.ttl
  8. +30 −30 test_data/ontologyR2RV2/boxOntologyWithValueObjects.ttl
  9. +75 −75 test_data/ontologyR2RV2/imagesBild.ttl
  10. +64 −64 test_data/ontologyR2RV2/incunabulaBook.ttl
  11. +76 −76 test_data/ontologyR2RV2/incunabulaOntologySimple.ttl
  12. +223 −223 test_data/ontologyR2RV2/incunabulaOntologyWithValueObjects.ttl
  13. +73 −73 test_data/ontologyR2RV2/incunabulaPage.ttl
  14. +142 −142 test_data/ontologyR2RV2/incunabulaPageAndBookWithValueObjects.ttl
  15. +33 −33 test_data/ontologyR2RV2/knoraApiDateValue.ttl
  16. +67 −0 test_data/ontologyR2RV2/knoraApiOntologySimple.jsonld
  17. +1,035 −704 test_data/ontologyR2RV2/knoraApiOntologySimple.rdf
  18. +131 −94 test_data/ontologyR2RV2/knoraApiOntologySimple.ttl
  19. +304 −0 test_data/ontologyR2RV2/knoraApiOntologyWithValueObjects.jsonld
  20. +2,000 −1,614 test_data/ontologyR2RV2/knoraApiOntologyWithValueObjects.rdf
  21. +1,283 −1,097 test_data/ontologyR2RV2/knoraApiOntologyWithValueObjects.ttl
  22. +92 −102 test_data/ontologyR2RV2/salsahGuiOntology.rdf
  23. +392 −392 test_data/ontologyR2RV2/standoffOntologyWithValueObjects.ttl
  24. +5 −2 webapi/src/main/resources/application.conf
  25. +20 −4 webapi/src/main/scala/org/knora/webapi/messages/OntologyConstants.scala
  26. +10 −0 webapi/src/main/scala/org/knora/webapi/messages/util/ConstructResponseUtilV2.scala
  27. +109 −0 webapi/src/main/scala/org/knora/webapi/messages/v2/responder/valuemessages/ValueMessagesV2.scala
  28. +9 −0 webapi/src/main/scala/org/knora/webapi/settings/KnoraSettings.scala
  29. +1 −0 webapi/src/test/scala/org/knora/webapi/BUILD.bazel
  30. +0 −1 webapi/src/test/scala/org/knora/webapi/e2e/v2/BUILD.bazel
  31. +1 −0 webapi/src/test/scala/org/knora/webapi/e2e/v2/OntologyV2R2RSpec.scala
  32. +187 −302 webapi/src/test/scala/org/knora/webapi/it/v2/KnoraSipiIntegrationV2ITSpec.scala
  33. +10 −0 webapi/src/test/scala/org/knora/webapi/models/BUILD.bazel
  34. +3 −5 webapi/src/test/scala/org/knora/webapi/{e2e/v2 → models}/OntologyModels.scala
  35. +26 −0 webapi/src/test/scala/org/knora/webapi/models/filemodels/BUILD.bazel
  36. +103 −0 webapi/src/test/scala/org/knora/webapi/models/filemodels/FileModelUtil.scala
  37. +374 −0 webapi/src/test/scala/org/knora/webapi/models/filemodels/FileModels.scala
  38. +171 −0 webapi/src/test/scala/org/knora/webapi/models/filemodels/FileModelsSpec.scala
  39. +81 −22 webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2Spec.scala

Large diffs are not rendered by default.

@@ -222,6 +222,7 @@ DSP-API v2 currently supports using Sipi to store the following types of files:
* Audio: MPEG, MP4, or Waveform audio file format (.wav, .x-wav, .vnd.wave)
* Text files: TXT, XML, or CSV
* Video files: MP4
* Archive files: ZIP, TAR, GZIP

Support for other types of files will be added in the future.

@@ -328,6 +329,10 @@ For a text file, use `knora-api:TextRepresentation`, which has the property
`knora-api:hasTextFileValue`, pointing to a
`knora-api:TextFileValue`.

For an archive like zip, use `knora-api:ArchiveRepresentation`, which has the property
`knora-api:hasArchiveFileValue`, pointing to a
`knora-api:ArchiveFileValue`.

## Updating a Value

To update a value, use this route:
@@ -371,22 +371,45 @@

:hasDocumentFileValue rdf:type owl:ObjectProperty ;

rdfs:label "hat Dokument"@de ,
"has document"@en ,
"a document"@fr ,
"ha documento"@it ;
rdfs:label "hat Dokument"@de ,
"has document"@en ,
"a document"@fr ,
"ha documento"@it ;

rdfs:comment "Connects a Representation to a document"@en ;
rdfs:comment "Connects a Representation to a document"@en ;

:objectClassConstraint :DocumentFileValue ;
:objectClassConstraint :DocumentFileValue ;

:subjectClassConstraint :DocumentRepresentation ;
:subjectClassConstraint :DocumentRepresentation ;

:isEditable true ;
:isEditable true ;

salsah-gui:guiElement salsah-gui:Fileupload ;
salsah-gui:guiElement salsah-gui:Fileupload ;

rdfs:subPropertyOf :hasFileValue .
rdfs:subPropertyOf :hasFileValue .



### http://www.knora.org/ontology/knora-base#hasArchiveFileValue

:hasArchiveFileValue rdf:type owl:ObjectProperty ;

rdfs:label "hat Zip"@de ,
"has zip"@en ,
"a zip"@fr ,
"ha zip"@it ;

rdfs:comment "Connects a Representation to a zip archive"@en ;

:objectClassConstraint :ArchiveFileValue ;

:subjectClassConstraint :ArchiveRepresentation ;

:isEditable true ;

salsah-gui:guiElement salsah-gui:Fileupload ;

rdfs:subPropertyOf :hasFileValue .



@@ -1041,8 +1064,8 @@
# No :subjectClassConstraint, because this can be used with :Resource or :Value.

:objectClassConstraint knora-admin:User .


#################################################################
#
# Datatype properties
@@ -1797,15 +1820,39 @@
:DocumentRepresentation rdf:type owl:Class ;

rdfs:label "Repräsentation (Dokument)"@de ,
"Representation (Document)"@en ,
"Répresentation (Document)"@fr ,
"Rappresentazione (Documento)"@it ;
"Representation (Document)"@en ,
"Répresentation (Document)"@fr ,
"Rappresentazione (Documento)"@it ;

rdfs:subClassOf :Representation ,
[ rdf:type owl:Restriction ;
owl:onProperty :hasDocumentFileValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
] .
[ rdf:type owl:Restriction ;
owl:onProperty :hasDocumentFileValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
] .



### http://www.knora.org/ontology/knora-base#ArchiveFileValue

:ArchiveFileValue rdf:type owl:Class ;

rdfs:subClassOf :FileValue .


### http://www.knora.org/ontology/knora-base#ArchiveRepresentation

:ArchiveRepresentation rdf:type owl:Class ;

rdfs:label "Repräsentation (Zip)"@de ,
"Representation (Zip)"@en ,
"Répresentation (Zip)"@fr ,
"Rappresentazione (Zip)"@it ;

rdfs:subClassOf :Representation ,
[ rdf:type owl:Restriction ;
owl:onProperty :hasArchiveFileValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
] .



@@ -11,6 +11,7 @@ IMAGE = "image"
DOCUMENT = "document"
AUDIO = "audio"
VIDEO = "video"
ARCHIVE = "archive"

-------------------------------------------------------------------------------
-- Mimetype constants
@@ -67,10 +68,7 @@ local text_mime_types = {

local document_mime_types = {
APPLICATION_PDF,
APPLICATION_TAR,
APPLICATION_ZIP,
APPLICATION_ISO,
APPLICATION_GZIP,
APPLICATION_DOC,
APPLICATION_DOCX,
APPLICATION_XLS,
@@ -79,6 +77,12 @@ local document_mime_types = {
APPLICATION_PPTX
}

local archive_mime_types = {
APPLICATION_TAR,
APPLICATION_ZIP,
APPLICATION_GZIP
}

local video_mime_types = {
VIDEO_MP4
}
@@ -99,10 +103,7 @@ local text_extensions = {

local document_extensions = {
"pdf",
"zip",
"tar",
"iso",
"gz",
"doc",
"docx",
"xls",
@@ -111,6 +112,12 @@ local document_extensions = {
"pptx"
}

local archive_extensions = {
"zip",
"tar",
"gz"
}

local video_extensions = {
"mp4"
}
@@ -162,6 +169,17 @@ function make_document_file_info(extension)
end
end

function make_archive_file_info(extension)
if not table.contains(archive_extensions, extension) then
return nil
else
return {
media_type = ARCHIVE,
extension = extension
}
end
end

-------------------------------------------------------------------------------
-- Determines the media type and file extension of a file.
-- Parameters:
@@ -174,7 +192,7 @@ end
function get_file_info(filename, mimetype)

local extension = filename:match("^.+%.([^.]+)$")

if extension == nil then
return nil
elseif table.contains(image_mime_types, mimetype) then
@@ -187,6 +205,8 @@ function get_file_info(filename, mimetype)
return make_text_file_info(extension)
elseif table.contains(document_mime_types, mimetype) then
return make_document_file_info(extension)
elseif table.contains(archive_mime_types, mimetype) then
return make_archive_file_info(extension)
else
-- no supported mediatype could be determined
return nil
@@ -41,7 +41,7 @@

:Book
rdf:type owl:Class ;
rdfs:subClassOf knora-base:Resource;
rdfs:subClassOf knora-base:Resource ;
knora-base:resourceIcon "book.png" ;
rdfs:label "Buch"@de, "Book"@en ;
rdfs:comment """A resource describing a book."""@en .
@@ -58,7 +58,7 @@

:Page
rdf:type owl:Class ;
rdfs:subClassOf knora-base:Resource;
rdfs:subClassOf knora-base:Resource ;
knora-base:resourceIcon "page.png" ;
rdfs:label "Seite"@de, "Page"@en ;
rdfs:comment """A resource describing a page in a book."""@en .

0 comments on commit 0123a8f

Please sign in to comment.