Skip to content
Permalink
Browse files
feat: return a DeletedResource or DeletedValue instead of 404 if a de…
…leted resource or value is requested (DEV-226) (#1960)

* feat: add DeletedValue and DeletedResource to base ontology; see how things work

* feat: return DeletedResource for deleted resources

* chore: update test data

* refactor: move toDeletedResource to message; feat: add deleted resource logic to preview

* chore: update client-test-data

* feat: implement DeletedValue

* fix: version history can be retrieved even if a value was deleted

* refactor: string interpolation issues

* feast: correctly handle DeletedValue in resource history

* chore: update test data

* refactor: get rid of some code smells

* refactor: remove additional code smell

* test: improve tests for deleted resources

* docs: add docstrings

* Update ValuesResponderV2Spec.scala

* docs: change codeblock language tag from jsonld to json

* docs: add documentation for deleted resource

* fix: actually return deleted values

* refactor: remove codesmell

* docs: document DeletedValue

* chore: update test data

* docs: rectify documentation on DeletedValue

* fix: correctly check if LinkValue was deleted

* docs: fix typos according to review

Co-authored-by: irinaschubert <irina.schubert@dasch.swiss>

* docs: apply changes according to review

Co-authored-by: Marcin Procyk <marcin.procyk@dasch.swiss>

* docs: make snippets in docs valid JSON

* refactor: rename some unclearly named variables

Co-authored-by: irinaschubert <irina.schubert@dasch.swiss>
Co-authored-by: Marcin Procyk <marcin.procyk@dasch.swiss>
  • Loading branch information
3 people committed Jan 5, 2022
1 parent af64021 commit c78e2522a96daf3fc52dc73d7daddf8f2a7d8c6a
Showing with 21,824 additions and 19,936 deletions.
  1. +117 −76 docs/03-apis/api-v2/editing-resources.md
  2. +283 −261 docs/03-apis/api-v2/editing-values.md
  3. +69 −63 docs/03-apis/api-v2/ontology-information.md
  4. +1 −1 docs/03-apis/api-v2/permalinks.md
  5. +5 −5 docs/03-apis/api-v2/reading-and-searching-resources.md
  6. +18 −0 knora-ontologies/knora-base.ttl
  7. +437 −437 test_data/ontologyR2RV2/anythingOntologyWithValueObjects.ttl
  8. +88 −88 test_data/ontologyR2RV2/anythingThingWithAllLanguages.ttl
  9. +29 −29 test_data/ontologyR2RV2/boxOntologyWithValueObjects.ttl
  10. +83 −83 test_data/ontologyR2RV2/imagesBild.ttl
  11. +66 −66 test_data/ontologyR2RV2/incunabulaBook.ttl
  12. +84 −84 test_data/ontologyR2RV2/incunabulaOntologySimple.ttl
  13. +220 −220 test_data/ontologyR2RV2/incunabulaOntologyWithValueObjects.ttl
  14. +76 −76 test_data/ontologyR2RV2/incunabulaPage.ttl
  15. +138 −138 test_data/ontologyR2RV2/incunabulaPageAndBookWithValueObjects.ttl
  16. +28 −28 test_data/ontologyR2RV2/knoraApiDateValue.ttl
  17. +52 −0 test_data/ontologyR2RV2/knoraApiOntologySimple.jsonld
  18. +186 −134 test_data/ontologyR2RV2/knoraApiOntologySimple.rdf
  19. +116 −85 test_data/ontologyR2RV2/knoraApiOntologySimple.ttl
  20. +264 −0 test_data/ontologyR2RV2/knoraApiOntologyWithValueObjects.jsonld
  21. +2,155 −1,811 test_data/ontologyR2RV2/knoraApiOntologyWithValueObjects.rdf
  22. +1,302 −1,138 test_data/ontologyR2RV2/knoraApiOntologyWithValueObjects.ttl
  23. +404 −404 test_data/ontologyR2RV2/standoffOntologyWithValueObjects.ttl
  24. +593 −572 test_data/resourcesR2RV2/BookReiseInsHeiligeLand.jsonld
  25. +386 −287 test_data/resourcesR2RV2/BookReiseInsHeiligeLand.rdf
  26. +270 −251 test_data/resourcesR2RV2/BookReiseInsHeiligeLand.ttl
  27. +31 −30 test_data/resourcesR2RV2/BookReiseInsHeiligeLandPreview.jsonld
  28. +52 −31 test_data/resourcesR2RV2/BookReiseInsHeiligeLandSimple.jsonld
  29. +38 −37 test_data/resourcesR2RV2/BookReiseInsHeiligeLandSimple.rdf
  30. +19 −21 test_data/resourcesR2RV2/BookReiseInsHeiligeLandSimple.ttl
  31. +19 −18 test_data/resourcesR2RV2/BookReiseInsHeiligeLandSimplePreview.jsonld
  32. +11,141 −10,786 test_data/resourcesR2RV2/BooksFromIncunabula.jsonld
  33. +89 −78 test_data/resourcesR2RV2/CompleteVersionHistory.jsonld
  34. +6 −5 test_data/resourcesR2RV2/IIIFManifest.jsonld
  35. +179 −178 test_data/resourcesR2RV2/NarrenschiffFirstPage.jsonld
  36. +71 −62 test_data/resourcesR2RV2/PartialVersionHistory.jsonld
  37. +431 −430 test_data/resourcesR2RV2/Testding.jsonld
  38. +153 −127 test_data/resourcesR2RV2/ThingGraphBoth.jsonld
  39. +116 −97 test_data/resourcesR2RV2/ThingGraphBothWithDepth.jsonld
  40. +102 −82 test_data/resourcesR2RV2/ThingGraphBothWithExcludedProp.jsonld
  41. +66 −57 test_data/resourcesR2RV2/ThingGraphInbound.jsonld
  42. +134 −110 test_data/resourcesR2RV2/ThingGraphOutbound.jsonld
  43. +59 −58 test_data/resourcesR2RV2/ThingWithBCEDate.jsonld
  44. +59 −58 test_data/resourcesR2RV2/ThingWithBCEDate2.jsonld
  45. +73 −72 test_data/resourcesR2RV2/ThingWithLinkComplex.jsonld
  46. +22 −21 test_data/resourcesR2RV2/ThingWithLinkSimple.jsonld
  47. +51 −50 test_data/resourcesR2RV2/ThingWithListValue.jsonld
  48. +23 −22 test_data/resourcesR2RV2/ThingWithListValueSimple.jsonld
  49. +73 −72 test_data/resourcesR2RV2/ThingWithOneDeletedResource.jsonld
  50. +51 −50 test_data/resourcesR2RV2/ThingWithOneHiddenResource.jsonld
  51. +51 −50 test_data/resourcesR2RV2/ThingWithTextLangComplex.jsonld
  52. +23 −22 test_data/resourcesR2RV2/ThingWithTextLangSimple.jsonld
  53. +99 −98 test_data/resourcesR2RV2/ThingWithVersionHistory.jsonld
  54. +39 −38 test_data/resourcesR2RV2/ThingWithVersionHistory20190208T150510Z.jsonld
  55. +61 −60 test_data/resourcesR2RV2/ThingWithVersionHistory20190210T100510Z.jsonld
  56. +79 −78 test_data/resourcesR2RV2/ThingWithVersionHistory20190210T103010Z.jsonld
  57. +99 −98 test_data/resourcesR2RV2/ThingWithVersionHistory20190211T090510Z.jsonld
  58. +99 −98 test_data/resourcesR2RV2/ThingWithVersionHistory20190211T100510Z.jsonld
  59. +99 −98 test_data/resourcesR2RV2/ThingWithVersionHistory20190212T090510Z.jsonld
  60. +99 −98 test_data/resourcesR2RV2/ThingWithVersionHistory20190212T100510Z.jsonld
  61. +106 −77 test_data/resourcesR2RV2/ThingWithVersionHistory20190213T090010Z.jsonld
  62. +106 −77 test_data/resourcesR2RV2/ThingWithVersionHistory20190213T090510Z.jsonld
  63. +1 −0 webapi/scripts/expected-client-test-data.txt
  64. +3 −0 webapi/src/main/scala/org/knora/webapi/messages/OntologyConstants.scala
  65. +2 −2 webapi/src/main/scala/org/knora/webapi/messages/util/ConstructResponseUtilV2.scala
  66. +65 −2 ...i/src/main/scala/org/knora/webapi/messages/v2/responder/resourcemessages/ResourceMessagesV2.scala
  67. +53 −0 webapi/src/main/scala/org/knora/webapi/messages/v2/responder/valuemessages/ValueMessagesV2.scala
  68. +47 −8 webapi/src/main/scala/org/knora/webapi/responders/v2/ResourcesResponderV2.scala
  69. +0 −1 ...rg/knora/webapi/messages/twirl/queries/sparql/v2/getResourcePropertiesAndValuesStandard.scala.txt
  70. +93 −77 webapi/src/test/scala/org/knora/webapi/e2e/v2/ResourcesRouteV2E2ESpec.scala
  71. +14 −13 webapi/src/test/scala/org/knora/webapi/responders/v2/ResourcesResponderV2Spec.scala
  72. +58 −52 webapi/src/test/scala/org/knora/webapi/responders/v2/ValuesResponderV2Spec.scala
@@ -14,15 +14,15 @@ HTTP POST to http://host/v2/resources
```

The body of the request is a JSON-LD document in the
[complex API schema](introduction.md#api-schema), specifying the type,`rdfs:label`, and its Knora resource properties and their values. The representation of the
resource is the same as when it is returned in a `GET` request, except that its `knora-api:attachedToUser` is not given,
and the resource IRI and those of its values can be optionally specified. The format of the values submitted
is described in [Editing Values](editing-values.md). If there are multiple values for a property,
these must be given in an array.
[complex API schema](introduction.md#api-schema), specifying the type,`rdfs:label`, and its Knora resource properties
and their values. The representation of the resource is the same as when it is returned in a `GET` request, except that
its `knora-api:attachedToUser` is not given, and the resource IRI and those of its values can be optionally specified.
The format of the values submitted is described in [Editing Values](editing-values.md). If there are multiple values for
a property, these must be given in an array.

For example, here is a request to create a resource with various value types:

```jsonld
```json
{
"@type" : "anything:Thing",
"anything:hasBoolean" : {
@@ -120,12 +120,12 @@ For example, here is a request to create a resource with various value types:
}
```

Permissions for the new resource can be given by adding `knora-api:hasPermissions`, a custom creation date
can be specified by adding `knora-api:creationDate`
(an [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp)), and the
resource's creator can be specfied by adding `knora-api:attachedToUser`. For example:
Permissions for the new resource can be given by adding `knora-api:hasPermissions`, a custom creation date can be
specified by adding `knora-api:creationDate`
(an [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp)), and the resource's creator can be specfied
by adding `knora-api:attachedToUser`. For example:

```jsonld
```json
{
"@type" : "anything:Thing",
"anything:hasBoolean" : {
@@ -157,36 +157,34 @@ resource's creator can be specfied by adding `knora-api:attachedToUser`. For exa
The format of the object of `knora-api:hasPermissions` is described in
[Permissions](../../02-knora-ontologies/knora-base.md#permissions).

If permissions are not given, configurable default permissions are used
(see [Default Object Access Permissions](../../05-internals/design/api-admin/administration.md#default-object-access-permissions)).
If permissions are not given, configurable default permissions are used (see
[Default Object Access Permissions](../../05-internals/design/api-admin/administration.md#default-object-access-permissions)
).

To create a resource, the user must have permission to create resources
of that class in that project.
To create a resource, the user must have permission to create resources of that class in that project.

The predicate `knora-api:attachedToUser` can be used to specify a creator other
than the requesting user only if the requesting user is an administrator of the
project or a system administrator. The specified creator must also
have permission to create resources of that class in that project.
The predicate `knora-api:attachedToUser` can be used to specify a creator other than the requesting user only if the
requesting user is an administrator of the project or a system administrator. The specified creator must also have
permission to create resources of that class in that project.

In addition to the creation date, in the body of the request, it is possible to specify a custom IRI (of [Knora IRI](knora-iris.md#iris-for-data) form) for a resource through
the `@id` attribute which will then be assigned to the resource; otherwise the resource will get a unique random IRI.
In addition to the creation date, in the body of the request, it is possible to specify a custom IRI (
of [Knora IRI](knora-iris.md#iris-for-data) form) for a resource through the `@id` attribute which will then be assigned
to the resource; otherwise the resource will get a unique random IRI.

A custom resource IRI must be `http://rdfh.ch/PROJECT_SHORTCODE/` (where `PROJECT_SHORTCODE`
is the shortcode of the project that the resource belongs to), plus a custom ID string.
is the shortcode of the project that the resource belongs to) plus a custom ID string.

Similarly, it is possible to assign a custom IRI to the values using their `@id` attributes; if not given, random IRIs
will be assigned to the values.

A custom value IRI must be the IRI of the containing resource, followed
by a `/values/` and a custom ID string.
A custom value IRI must be the IRI of the containing resource, followed by a `/values/` and a custom ID string.

An optional custom UUID of a value can also be given by adding `knora-api:valueHasUUID`.
Each custom UUID must be [base64url-encoded](https://tools.ietf.org/html/rfc4648#section-5), without padding. Each value of the new resource
can also have a custom creation date specified by adding `knora-api:creationDate`
(an [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp)).
For example:
An optional custom UUID of a value can also be given by adding `knora-api:valueHasUUID`. Each custom UUID must
be [base64url-encoded](https://tools.ietf.org/html/rfc4648#section-5) without padding. Each value of the new resource
can also have a custom creation date specified by adding `knora-api:creationDate`
(an [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp)). For example:

```jsonld
```json
{
"@id" : "http://rdfh.ch/0001/oveR1dQltEUwNrls9Lu5Rw",
"@type" : "anything:Thing",
@@ -217,7 +215,7 @@ For example:
}
}
```

The response is a JSON-LD document containing a
[preview](reading-and-searching-resources.md#get-the-preview-of-a-resource-by-iri)
of the resource.
@@ -244,17 +242,21 @@ The request body is a JSON-LD object containing the following information about

- `@id`: the resource's IRI
- `@type`: the resource's class IRI
- `knora-api:lastModificationDate`: an `xsd:dateTimeStamp` representing the last modification date that is currently attached to the resource, if any. This is used to make sure that the resource has not been modified by someone else since you last read it.
- `knora-api:lastModificationDate`: an `xsd:dateTimeStamp` representing the last modification date that is currently
attached to the resource, if any. This is used to make sure that the resource has not been modified by someone else
since you last read it.

The submitted JSON-LD object must also contain one or more of the following predicates, representing the metadata you want to change:
The submitted JSON-LD object must also contain one or more of the following predicates, representing the metadata you
want to change:

- `rdfs:label`: a string
- `knora-api:hasPermissions`, in the format described in [Permissions](../../02-knora-ontologies/knora-base.md#permissions)
- `knora-api:hasPermissions`, in the format described
in [Permissions](../../02-knora-ontologies/knora-base.md#permissions)
- `knora-api:newModificationDate`: an [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp).

Here is an example:

```jsonld
```json
{
"@id" : "http://rdfh.ch/0001/a-thing",
"@type" : "anything:Thing",
@@ -278,8 +280,8 @@ Here is an example:
}
```

If you submit a `knora-api:lastModificationDate` that is different from the resource's actual last modification
date, you will get an HTTP 409 (Conflict) error.
If you submit a `knora-api:lastModificationDate` that is different from the resource's actual last modification date,
you will get an HTTP 409 (Conflict) error.

If you submit a `knora-api:newModificationDate` that is earlier than the resource's `knora-api:lastModificationDate`,
you will get an HTTP 400 (Bad Request) error.
@@ -288,8 +290,8 @@ A successful response is an HTTP 200 (OK) status containing the resource's metad

## Deleting a Resource

Knora does not normally delete resources; instead, it marks them as deleted, which means
that they do not appear in normal query results.
Knora does not normally delete resources; instead, it marks them as deleted, which means that they do not appear in
normal query results.

To mark a resource as deleted, use this route:

@@ -301,9 +303,11 @@ The request body is a JSON-LD object containing the following information about

- `@id`: the resource's IRI
- `@type`: the resource's class IRI
- `knora-api:lastModificationDate`: an `xsd:dateTimeStamp` representing the last modification date that is currently attached to the resource, if any. This is used to make sure that the resource has not been modified by someone else since you last read it.
- `knora-api:lastModificationDate`: an `xsd:dateTimeStamp` representing the last modification date that is currently
attached to the resource, if any. This is used to make sure that the resource has not been modified by someone else
since you last read it.

```jsonld
```json
{
"@id" : "http://rdfh.ch/0001/a-thing",
"@type" : "anything:Thing",
@@ -322,56 +326,93 @@ The request body is a JSON-LD object containing the following information about
}
```

The optional property `knora-api:deleteComment` specifies a comment to be attached to the
resource, explaining why it has been marked as deleted.
The optional property `knora-api:deleteComment` specifies a comment to be attached to the resource, explaining why it
has been marked as deleted.

The optional property `knora-api:deleteDate`
(an [xsd:dateTimeStamp](https://www.w3.org/TR/xmlschema11-2/#dateTimeStamp))
indicates when the resource was marked as deleted; if not given, the current
time is used.
indicates when the resource was marked as deleted; if not given, the current time is used.

The response is a JSON-LD document containing the predicate `knora-api:result` with a confirmation message.

### Requesting Deleted Resources

Resources marked as deleted are not found in search queries. It is however possible to request them directly or from an
ARK URL. In these instances, the API will not return the deleted resource, but instead a generic resource of type
`knora-base:DeletedResource`. This resource will be similar to the requested resource, having e.g. the same IRI.
The resource will contain the deletion date and optionally the deletion comment.

The response is a JSON-LD document containing the predicate `knora-api:result`
with a confirmation message.
The response to requesting a deleted resource will look as the following example:

```json
{
"rdfs:label": "Deleted Resource",
"knora-api:versionArkUrl": {
"@value": "http://0.0.0.0:3336/ark:/72163/1/0001/a=thingO.20211214T084407677335Z",
"@type": "xsd:anyURI"
},
"knora-api:attachedToProject": {
"@id": "http://rdfh.ch/projects/0001"
},
"knora-api:userHasPermission": "CR",
"knora-api:attachedToUser": {
"@id": "http://rdfh.ch/users/9XBCrDV3SRa7kS1WwynB4Q"
},
"knora-api:hasPermissions": "CR knora-admin:ProjectMember|V knora-admin:ProjectMember",
"knora-api:isDeleted": true,
"@type": "knora-api:DeletedResource",
"@id": "http://rdfh.ch/0001/a-thing",
"knora-api:deleteComment": "This resource is too boring.",
"knora-api:arkUrl": {
"@value": "http://0.0.0.0:3336/ark:/72163/1/0001/a=thingO",
"@type": "xsd:anyURI"
},
"knora-api:creationDate": {
"@value": "2021-12-14T08:44:07.677335Z",
"@type": "xsd:dateTimeStamp"
},
"knora-api:deleteDate": {
"@type": "xsd:dateTimeStamp",
"@value": "2021-12-14T08:44:07.372543Z"
},
"@context": {
"rdf": "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
"rdfs": "http://www.w3.org/2000/01/rdf-schema#",
"xsd": "http://www.w3.org/2001/XMLSchema#",
"knora-api": "http://api.knora.org/ontology/knora-api/v2#"
}
}
```

### Links to Deleted Resources

If resource `A` has a link to resource `B`, and resource
`B` is later marked as deleted, `A`'s link will still exist. DSP-API
v2 will still return the link when `A` is queried, but without any information
about `B` (except for `B`'s IRI). If `A`'s link is necessary to
meet the requirements of a cardinality, marking `B` as deleted will
not violate the cardinality.

The reason for this design is that `A` and `B` might be in
different projects, and each project must retain control of its resources
and be able to mark them as deleted, even if they are used by another
project. In future, Knora may be able to notify the owner of `A` in this
case.
`B` is later marked as deleted, `A`'s link will still exist. DSP-API v2 will still return the link when `A` is queried,
but without any information about `B` (except for `B`'s IRI). If `A`'s link is necessary to meet the requirements of a
cardinality, marking `B` as deleted will not violate the cardinality.

The reason for this design is that `A` and `B` might be in different projects, and each project must retain control of
its resources and be able to mark them as deleted, even if they are used by another project.

## Erasing a Resource from the Triplestore

Normally, resources are not actually removed from the triplestore; they are
only marked as deleted (see [Deleting a Resource](#deleting-a-resource)).
However, sometimes it is necessary to erase a resource from the triplestore.
To do so, use this route:
Normally, resources are not actually removed from the triplestore; they are only marked as deleted (see
[Deleting a Resource](#deleting-a-resource)). However, sometimes it is necessary to erase a resource from the
triplestore. To do so, use this route:

```
HTTP POST to http://host/v2/resources/erase
```

The request body is the same as for [Deleting a Resource](#deleting-a-resource),
except that `knora-api:deleteComment` is not relevant and will be ignored.
The request body is the same as for [Deleting a Resource](#deleting-a-resource), except that `knora-api:deleteComment`
is not relevant and will be ignored.

To do this, a user must be a system administrator or an administrator of the
project containing the resource. The user's permissions on the resource are
not otherwise checked.
To do this, a user must be a system administrator or an administrator of the project containing the resource. The user's
permissions on the resource are not otherwise checked.

A resource cannot be erased if any other resource has a link to it. Any such
links must first be changed or marked as deleted
(see [Updating a Value](editing-values.md#updating-a-value) and
[Deleting a Value](editing-values.md#deleting-a-value)). Then,
when the resource is erased, the deleted link values that referred to
it will also be erased.
A resource cannot be erased if any other resource has a link to it. Any such links must first be changed or marked as
deleted (see [Updating a Value](editing-values.md#updating-a-value) and
[Deleting a Value](editing-values.md#deleting-a-value)). Then, when the resource is erased, the deleted link values that
referred to it will also be erased.

This operation cannot be undone (except by restoring the repository from a
backup), so use it with care.
This operation cannot be undone (except by restoring the repository from a backup), so use it with care.

0 comments on commit c78e252

Please sign in to comment.