Skip to content
Permalink
Browse files
fix(authentication)!: add server specific issuer to JWT token (DEV-555)…
… (#2024)

* add issuer to token

* Update docstring with issuer

* check token issuer for sipi

* remove knora from sipi config file names

* update token string in test

* add test for different issuer

* update documentation

* rename test

* define new env variables for sipi

* improve error message
  • Loading branch information
irinaschubert committed Mar 23, 2022
1 parent 747d13d commit 4bd5b2fccd8ef34c5ff8240269e3c290ab106850
@@ -125,7 +125,7 @@ lazy val webapi: Project = Project(id = "webapi", base = file("webapi"))
// add needed files to test jar
Test / packageBin / mappings ++= Seq(
(rootBaseDir.value / "webapi" / "scripts" / "fuseki-repository-config.ttl.template") -> "webapi/scripts/fuseki-repository-config.ttl.template", // needed for initialization of triplestore
(rootBaseDir.value / "sipi" / "config" / "sipi.knora-docker-config.lua") -> "sipi/config/sipi.knora-docker-config.lua"
(rootBaseDir.value / "sipi" / "config" / "sipi.docker-config.lua") -> "sipi/config/sipi.docker-config.lua"
),
// use packaged jars (through packageBin) on classpaths instead of class directories for test
Test / exportJars := true
@@ -33,8 +33,10 @@ services:
- SIPI_EXTERNAL_PORT=1024
- SIPI_WEBAPI_HOSTNAME=api
- SIPI_WEBAPI_PORT=3333
- KNORA_WEBAPI_KNORA_API_EXTERNAL_HOST=0.0.0.0
- KNORA_WEBAPI_KNORA_API_EXTERNAL_PORT=3333
# entrypoint: [ "valgrind", "--leak-check=yes", "/sipi/sipi" ] ## uncomment to run SIPI under valgrind
command: --config=/sipi/config/sipi.knora-docker-config.lua
command: --config=/sipi/config/sipi.docker-config.lua

api:
image: daschswiss/knora-api:latest
@@ -58,6 +60,8 @@ services:
- KNORA_WEBAPI_CACHE_SERVICE_REDIS_HOST=redis
- KNORA_WEBAPI_CACHE_SERVICE_REDIS_PORT=6379
- KNORA_WEBAPI_ALLOW_RELOAD_OVER_HTTP=true
- KNORA_WEBAPI_KNORA_API_EXTERNAL_HOST=0.0.0.0
- KNORA_WEBAPI_KNORA_API_EXTERNAL_PORT=3333

networks:
knora-net:
@@ -3,24 +3,24 @@
* SPDX-License-Identifier: Apache-2.0
-->

# Knora and Sipi
# DSP-API and Sipi

## Configuration

The Knora-specific configuration and scripts for Sipi are in the
`sipi` subdirectory of the Knora source tree. See the `README.md` there for
instructions on how to start Sipi with Knora.
The DSP-API specific configuration and scripts for Sipi are in the
`sipi` subdirectory of the DSP-API source tree. See the `README.md` for
instructions on how to start Sipi with DSP-API.

## Lua Scripts

DSP-API v2 uses custom Lua scripts to control Sipi. These scripts can be
found in `sipi/scripts` in the Knora source tree.
found in `sipi/scripts` in the DSP-API source tree.

Each of these scripts expects a [JSON Web Token](https://jwt.io/) in the
URL parameter `token`. In all cases, the token must be signed by Knora,
it must have an expiration date and not have expired, its issuer must be `Knora`,
and its audience must include `Sipi`. The other contents of the expected tokens
are described below.
URL parameter `token`. In all cases, the token must be signed by DSP-API,
it must have an expiration date and not have expired, its issuer must equal
the hostname and port of the API, and its audience must include `Sipi`.
The other contents of the expected tokens are described below.

### upload.lua

@@ -53,7 +53,7 @@ must be a JSON object containing:
### delete_temp_file.lua

The `delete_temp_file.lua` script is available at Sipi's `delete_temp_file` route.
It is used only if Knora rejects a file value update request. It expects an
It is used only if DSP-API rejects a file value update request. It expects an
HTTP `DELETE` request, with a filename as the last component of the URL.

The JWT sent to this script must contain the key `knora-data`, whose value
@@ -64,7 +64,7 @@ must be a JSON object containing:

## SipiConnector

In Knora, the `org.knora.webapi.iiif.SipiConnector` handles all communication
In DSP-API, the `org.knora.webapi.iiif.SipiConnector` handles all communication
with Sipi. It blocks while processing each request, to ensure that the number of
concurrent requests to Sipi is not greater than
`akka.actor.deployment./storeManager/iiifManager/sipiConnector.nr-of-instances`.
@@ -76,7 +76,7 @@ If it encounters an error, it returns `SipiException`.
`upload.lua`. The image is converted to JPEG 2000 and stored in Sipi's `tmp`
directory. In the response, the client receives the JPEG 2000's unique,
randomly generated filename.
2. The client submits a JSON-LD request to a Knora route (`/v2/values` or `/v2/resources`)
2. The client submits a JSON-LD request to a DSP-API route (`/v2/values` or `/v2/resources`)
to create or change a file value. The request includes Sipi's internal filename.
3. During parsing of this JSON-LD request, a `StillImageFileValueContentV2`
is constructed to represent the file value. During the construction of this
@@ -93,14 +93,14 @@ If it encounters an error, it returns `SipiException`.
`DeleteTemporaryFileRequestV2` to `SipiConnector`, which makes a request
to Sipi's `delete_temp_file` route.

If the request to Knora cannot be parsed, the temporary file is not deleted
If the request to DSP-API cannot be parsed, the temporary file is not deleted
immediately, but it will be deleted during the processing of a subsequent
request by Sipi's `upload` route.

If Sipi's `store` route fails, Knora returns the `SipiException` to the client.
If Sipi's `store` route fails, DSP-API returns the `SipiException` to the client.
In this case, manual intervention may be necessary to restore consistency
between Knora and Sipi.
between DSP-API and Sipi.

If Sipi's `delete_temp_file` route fails, the error is not returned to the client,
because there is already a Knora error that needs to be returned to the client.
because there is already a DSP-API error that needs to be returned to the client.
In this case, the Sipi error is simply logged.
@@ -78,7 +78,7 @@ image in the background. The default behaviour is to start Sipi by
calling the following command:

```
$ /sipi/local/bin/sipi -config /sipi/config/sipi.knora-test-config.lua
$ /sipi/local/bin/sipi -config /sipi/config/sipi.test-config.lua
```

To override this default behaviour, start the container by supplying
@@ -101,7 +101,7 @@ $ docker run --name sipi \
-p 1024:1024 \
-v $PWD:/localdir \
daschswiss/sipi \
/sipi/local/bin/sipi -config /localdir/sipi.knora-test-config.lua
/sipi/local/bin/sipi -config /localdir/sipi.test-config.lua
```

## Redis Server
@@ -9,8 +9,8 @@
for serving and converting binary media files such as images and video. Sipi can
efficiently convert between many different formats on demand, preserving
embedded metadata, and implements the [International Image
Interoperability Framework (IIIF)](http://iiif.io/). Knora is designed
Interoperability Framework (IIIF)](http://iiif.io/). DSP-API is designed
to use Sipi for converting and serving media files.

* [Setting Up Sipi for Knora](setup-sipi-for-knora.md)
* [Interaction Between Sipi and Knora](sipi-and-knora.md)
* [Setting Up Sipi for DSP-API](setup-sipi-for-dsp-api.md)
* [Interaction Between Sipi and DSP-API](sipi-and-dsp-api.md)
@@ -3,65 +3,64 @@
* SPDX-License-Identifier: Apache-2.0
-->

# Setting Up Sipi for Knora
# Setting Up Sipi for DSP-API

## Setup and Execution

In order to serve files to the client application like the Salsah GUI,
Sipi must be set up and running. Sipi can be downloaded from its own
GitHub repository: <https://github.com/dhlab-basel/Sipi> (which requires
GitHub repository: <https://github.com/dasch-swiss/sipi> (which requires
building from source), or the published [docker image](https://hub.docker.com/r/daschswiss/sipi/).
can be used. To start Sipi, run the following command from inside the `sipi/`
folder:

```
$ export DOCKERHOST=LOCAL_IP_ADDRESS
$ docker image rm --force daschswiss/sipi:main // deletes cached image and needs only to be used when newer image is available on dockerhub
$ docker run --rm -it --add-host webapihost:$DOCKERHOST -v $PWD/config:/sipi/config -v $PWD/scripts:/sipi/scripts -v /tmp:/tmp -v $HOME:$HOME -p 1024:1024 daschswiss/sipi:main --config=/sipi/config/sipi.knora-docker-config.lua
$ docker run --rm -it --add-host webapihost:$DOCKERHOST -v $PWD/config:/sipi/config -v $PWD/scripts:/sipi/scripts -v /tmp:/tmp -v $HOME:$HOME -p 1024:1024 daschswiss/sipi:main --config=/sipi/config/sipi.docker-config.lua
```

where `LOCAL_IP_ADDRESS` is the IP of the host running the `Knora`.
where `LOCAL_IP_ADDRESS` is the IP of the host running `DSP-API`.

`--config=/sipi/config/sipi.knora-docker-config.lua` (or `--config=/sipi/config/sipi.knora-docker-it-config.lua` for
using sipi for integration testing). Please see `sipi.knora-docker-config.lua` for the settings like URL, port number
etc. These settings need to be set accordingly in Knora's `application.conf`. If you use the default settings both in
Sipi and Knora, there is no need to change these settings.
`--config=/sipi/config/sipi.docker-config.lua`. Please see `sipi.docker-config.lua` for the settings like URL, port number
etc. These settings need to be set according to DSP-API's `application.conf`. If you use the default settings both in
Sipi and DSP-API, there is no need to change these settings.

Whenever a file is requested from Sipi (e.g. a browser trying to
dereference an image link served by Knora), a preflight function is
called. This function is defined in `sipi.init-knora.lua` present in the
dereference an image link served by DSP-API), a preflight function is
called. This function is defined in `sipi.init.lua` present in the
Sipi root directory. It takes three parameters: `prefix`, `identifier`
(the name of the requested file), and `cookie`. The prefix is the shortcode
of the project that the resource containing the file value belongs to.

Given this information, Sipi asks Knora about the current's users
permissions on the given file. The cookie contains the current user's
Knora session id, so Knora can match Sipi's request with a given user
Given this information, Sipi asks the API about the current user's
permissions on the given file. The cookie contains the current user's
session id, so the API can match Sipi's request with a given user
profile and determine the permissions this user has on the file. If the
Knora response grants sufficient permissions, the file is served in the
requested quality. If the suer has preview rights, Sipi serves a reduced
response grants sufficient permissions, the file is served in the
requested quality. If the user has preview rights, Sipi serves the file in reduced
quality or integrates a watermark. If the user has no permissions, Sipi
refuses to serve the file. However, all of this behaviour is defined in
the preflight function in Sipi and not controlled by Knora. Knora only
the preflight function in Sipi and not controlled by the API. DSP-API only
provides the permission code.

See [Authentication of Users with Sipi](sipi-and-knora.md#authentication-of-users-with-sipi) for more
See [Authentication of Users with Sipi](sipi-and-dsp-api.md#authentication-of-users-with-sipi) for more
information about sharing the session ID.

## Using Sipi in Test Mode

If you just want to test Sipi with Knora without serving the actual
If you just want to test Sipi with DSP-API without serving the actual
files (e.g. when executing browser tests), you can simply start Sipi
like this:

```
$ export DOCKERHOST=LOCAL_IP_ADDRESS
$ docker image rm --force daschswiss/sipi:main // deletes cached image and needs only to be used when newer image is available on dockerhub
$ docker run --rm -it --add-host webapihost:$DOCKERHOST -v $PWD/config:/sipi/config -v $PWD/scripts:/sipi/scripts -v /tmp:/tmp -v $HOME:$HOME -p 1024:1024 daschswiss/sipi:main --config=/sipi/config/sipi.knora-docker-test-config.lua
$ docker run --rm -it --add-host webapihost:$DOCKERHOST -v $PWD/config:/sipi/config -v $PWD/scripts:/sipi/scripts -v /tmp:/tmp -v $HOME:$HOME -p 1024:1024 daschswiss/sipi:main --config=/sipi/config/sipi.docker-test-config.lua
```

Then always the same test file will be served which is included in Sipi. In test mode, Sipi will
not ask Knora about the user's permission on the requested file.
Then always the same test file will be served which is delivered with Sipi. In test mode, Sipi will
not ask DSP-API about the user's permission on the requested file.

## Additional Sipi Environment Variables

@@ -70,7 +69,7 @@ Additionally, these environment variables can be used to further configure Sipi:
- `SIPI_WEBAPI_HOSTNAME=localhost`: overrides `knora_path` in Sipi's config
- `SIPI_WEBAPI_PORT=3333`: overrides `knora_port` in Sipi's config

These variables need to be explicitly used like in `sipi.ini-knora.lua`:
These variables need to be explicitly used like in `sipi.init.lua`:

```lua
--
@@ -3,29 +3,29 @@
* SPDX-License-Identifier: Apache-2.0
-->

# Interaction Between Sipi and Knora
# Interaction Between Sipi and DSP-API

## General Remarks

Knora and Sipi (Simple Image Presentation Interface) are two
**complementary** software projects. Whereas Knora deals with data that
DSP-API and Sipi (Simple Image Presentation Interface) are two
**complementary** software projects. Whereas DSP-API deals with data that
is written to and read from a triplestore (metadata and annotations),
Sipi takes care of storing, converting and serving image files as well
as other types of files such as audio, video, or documents (binary files
it just stores and serves).

Knora and Sipi stick to a clear division of responsibility regarding
files: Knora knows about the names of files that are attached to
DSP-API and Sipi stick to a clear division of responsibility regarding
files: DSP-API knows about the names of files that are attached to
resources as well as some metadata and is capable of creating the URLs
for the client to request them from Sipi, but the whole handling of
files (storing, naming, organization of the internal directory
structure, format conversions, and serving) is taken care of by Sipi.

## Adding Files to Knora
## Adding Files to DSP

A file is first uploaded to Sipi, then its metadata is submitted to
Knora. The implementation of this procedure is described in
[Knora and Sipi](../05-internals/design/api-v2/sipi.md). Instructions
DSP. The implementation of this procedure is described in
[DSP-API and Sipi](../05-internals/design/api-v2/sipi.md). Instructions
for the client are given in
[Creating File Values](../03-apis/api-v2/editing-values.md#creating-file-values)
(for DSP-API v2) and in
@@ -44,7 +44,7 @@ different IIIF URLs, e.g. at different resolutions. See the `knora-api` ontology

### File URLs in API v1

In API v1, for each file value, Knora creates several Sipi URLs for accessing the file at different
In API v1, for each file value, DSP-API creates several Sipi URLs for accessing the file at different
resolutions:

```
@@ -91,10 +91,10 @@ Sipi, obtaining the binary representation in the desired quality.

## Authentication of Users with Sipi

Whenever a file is requested, Sipi asks Knora about the current user's permissions on the given file.
This is achieved by sharing the Knora session cookie with Sipi. When the user logs in to Knora using his
Whenever a file is requested, Sipi asks the DSP-API about the current user's permissions on the given file.
This is achieved by sharing the session cookie with Sipi. When the user logs in to DSP using his
browser (using either `V1` or `V2` authentication route), a session cookie containing a JWT token representing
the user is stored in the user's client. This session cookie is then read by Sipi and used to ask Knora for
the user is stored in the user's client. This session cookie is then read by Sipi and used to ask DSP-API for
the user's image permissions.

For the session cookie to be sent to Sipi, both the DSP-API and Sipi endpoints need to
@@ -121,8 +121,8 @@ nav:
# - Index: 06-salsah/index.md
- SIPI:
- Index: 07-sipi/index.md
- Setting Up Sipi for DSP-API: 07-sipi/setup-sipi-for-knora.md
- Interaction between Sipi and DSP-API: 07-sipi/sipi-and-knora.md
- Setting Up Sipi for DSP-API: 07-sipi/setup-sipi-for-dsp-api.md
- Interaction between Sipi and DSP-API: 07-sipi/sipi-and-dsp-api.md
- Lucene:
- Overview: 08-lucene/index.md
- Lucene Query Parser Syntax: 08-lucene/lucene-query-parser-syntax.md
@@ -96,7 +96,7 @@ sipi = {
--
-- Lua script which is executed on initialization of the Lua interpreter
--
initscript = '/sipi/scripts/sipi.init-knora.lua',
initscript = '/sipi/scripts/sipi.init.lua',

--
-- path to the caching directory
@@ -195,24 +195,6 @@ routes = {
method = 'DELETE',
route = '/delete_temp_file',
script = 'delete_temp_file.lua'
},
--
-- additional routes used for testing. should not be defined in production.
--
{
method = 'GET',
route = '/test_functions',
script = 'test_functions.lua'
},
{
method = 'GET',
route = '/test_file_info',
script = 'test_file_info.lua'
},
{
method = 'GET',
route = '/test_knora_session_cookie',
script = 'test_knora_session_cookie.lua'
}

}
@@ -49,7 +49,7 @@ sipi = {
--
-- Lua script which is executed on initialization of the Lua interpreter
--
initscript = '/sipi/scripts/sipi.init-knora-no-auth.lua',
initscript = '/sipi/scripts/sipi.init-no-auth.lua',

--
-- path to the caching directory
@@ -68,7 +68,7 @@ sipi = {
--
-- Lua script which is executed on initialization of the Lua interpreter
--
initscript = './scripts/sipi.init-knora-test.lua',
initscript = './scripts/sipi.init-test.lua',

--
-- path to the caching directory
@@ -68,7 +68,7 @@ sipi = {
--
-- Lua script which is executed on initialization of the Lua interpreter
--
initscript = './scripts/sipi.init-knora.lua',
initscript = './scripts/sipi.init.lua',

--
-- path to the caching directory
@@ -109,7 +109,7 @@ sipi = {
--
-- Path to Knora Application
--
knora_path = 'localhost',
knora_path = '0.0.0.0',

--
-- Port of Knora Application

0 comments on commit 4bd5b2f

Please sign in to comment.