Skip to content
No description, website, or topics provided.
Python Shell Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
rest
tests
workers
.env.example
.gitignore
.gitlab-ci.yml
README.md
config.toml
docker-compose.pytest.yml
docker-compose.yml
download-process-definitions.sh

README.md

Running locally

Mandatory: whichever method you select, first copy .env.example to .env and enter the variables there.

First, start up all services:

$ docker-compose up -d

Download definitions of defined processes:

$ ./download-process-definitions.sh

Then, python libraries need to be installed:

$ cd rest/
$ pipenv install

Tables on DynamoDB need to be created manually:

$ pipenv shell
<pipenv> $ python dynamodb.py

Then the REST API server can be run:

<pipenv> $ python app.py

Examples

REST

Jobs

List all jobs:

$ curl http://localhost:5000/jobs/

Create a job:

POST /jobs HTTP/1.1
Content-Type: application/json

{
  "process_graph": {
    "loadco1": {
      "process_id": "load_collection",
      "arguments": {
        "id": "S2L1C",
        "spatial_extent": {
          "west": 12.32271,
          "east": 12.33572,
          "north": 42.07112,
          "south": 42.06347
        },
        "temporal_extent": ["2019-08-16", "2019-08-18"]
      }
    },
    "ndvi1": {
      "process_id": "ndvi",
      "arguments": {
        "data": {"from_node": "loadco1"}
      }
    },
    "result1": {
      "process_id": "save_result",
      "arguments": {
        "data": {"from_node": "ndvi1"},
        "format": "gtiff"
      },
      "result": true
    }
  }
}

Using curl:

$ curl -i -X POST -H "Content-Type: application/json" -d '{"process_graph": {"loadco1": {"process_id": "load_collection", "arguments": {"id": "S2L1C", "temporal_extent": ["2019-08-16", "2019-08-18"], "spatial_extent": {"west": 12.32271, "east": 12.33572, "north": 42.07112, "south": 42.06347}}}, "ndvi1": {"process_id": "ndvi", "arguments": {"data": {"from_node": "loadco1"}}}, "result1": {"process_id": "save_result", "arguments": {"data": {"from_node": "ndvi1"}, "format": "gtiff"}, "result": true}}}' http://localhost:5000/jobs/

Listing all jobs should now include the new job: (note that id will be different)

$ curl http://localhost:5000/jobs/
{
  "jobs": [
    {
      "description": null,
      "id": "6520894b-d41d-40d1-bcff-67eafab4eced",
      "title": null
    }
  ],
  "links": [
    {
      "href": "/jobs/6520894b-d41d-40d1-bcff-67eafab4eced",
      "title": null
    }
  ]
}

Taking the job id, we can check job details:

$ curl http://localhost:5000/jobs/6520894b-d41d-40d1-bcff-67eafab4eced
{
  "description": null,
  "error": [
    null
  ],
  "id": "6520894b-d41d-40d1-bcff-67eafab4eced",
  "process_graph": {
    "loadco1": {
      "arguments": {
        "id": "S2L1C",
        "spatial_extent": {
          "east": 12.33572,
          "north": 42.07112,
          "south": 42.06347,
          "west": 12.32271
        },
        "temporal_extent": ["2019-08-16", "2019-08-18"]
      },
      "process_id": "load_collection"
    },
    "ndvi1": {
      "arguments": {
        "data": {
          "from_node": "loadco1"
        }
      },
      "process_id": "ndvi"
    },
    "result1": {
      "arguments": {
        "data": {
          "from_node": "ndvi1"
        },
        "format": "gtiff"
      },
      "process_id": "save_result",
      "result": true
    }
  },
  "status": "submitted",
  "submitted": [
    "2019-08-30T09:18:12.250595+00:00"
  ],
  "title": null
}

And start it by using the returned id:

$ curl -X POST http://localhost:5000/jobs/6520894b-d41d-40d1-bcff-67eafab4eced/results

Services

Get supported service types:

$ curl http://localhost:5000/service_types

Currently, only XYZ service type is supported. We create the service:

$ curl -i http://localhost:5000/services' -H 'Content-Type: application/json;charset=utf-8' -d '{"title":null,"description":null,"process_graph":{"loadco1":{"process_id":"load_collection","arguments":{"id":"S2L1C","spatial_extent":{"west":{"variable_id":"spatial_extent_west"},"east":{"variable_id":"spatial_extent_east"},"north":{"variable_id":"spatial_extent_north"},"south":{"variable_id":"spatial_extent_south"}},"temporal_extent":["2019-08-01","2019-08-18"],"bands":["B04","B08"],"options":{"width":256,"height":256}}},"ndvi1":{"process_id":"ndvi","arguments":{"data":{"from_node":"loadco1"}}},"reduce1":{"process_id":"reduce","arguments":{"data":{"from_node":"ndvi1"},"reducer":{"callback":{"2":{"process_id":"mean","arguments":{"data":{"from_argument":"data"}},"result":true}}},"dimension":"t"}},"linear1":{"process_id":"linear_scale_range","arguments":{"x":{"from_node":"reduce1"},"inputMin":0,"inputMax":1,"outputMax":255}},"result1":{"process_id":"save_result","arguments":{"data":{"from_node":"linear1"},"format":"JPEG","options":{"datatype":"byte"}},"result":true}},"type":"XYZ","enabled":true,"parameters":{},"plan":null,"budget":null}'
...
Location: http://localhost:5000/services/df9cf197-9125-4db6-a684-e9e02678cb4b
OpenEO-Identifier: df9cf197-9125-4db6-a684-e9e02678cb4b
...

As with jobs, IDs will be different. Then we can issue requests using the returned service id:

$ curl http://localhost:5000/service/xyz/df9cf197-9125-4db6-a684-e9e02678cb4b/15/17194/11145 > /tmp/test.jpeg

AWS

Since we are running everything locally, make sure that the default entry in ~/.aws/credentials has correct credentials:

[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

While executing the examples, we can check the data of the jobs in DynamoDB by using AWS CLI tool:

$ aws dynamodb --endpoint http://localhost:8000 scan --table-name shopeneo_jobs

Similarly, S3 can be inspected:

$ aws s3 --endpoint http://localhost:9000 ls
2019-08-30 12:44:38 com.sinergise.openeo.results
$ aws s3 --endpoint http://localhost:9000 ls s3://com.sinergise.openeo.results
                    PRE 41a32a56-29bd-42d7-9c27-5af98a3421a8/
$ aws s3 --endpoint http://localhost:9000 ls s3://com.sinergise.openeo.results/41a32a56-29bd-42d7-9c27-5af98a3421a8/
2019-08-30 12:44:38      73206 result-0.tiff

And results downloaded to local filesystem if needed:

$ mkdir /tmp/results
$ aws s3 --endpoint http://localhost:9000 sync s3://com.sinergise.openeo.results/41a32a56-29bd-42d7-9c27-5af98a3421a8/ /tmp/results
download: s3://com.sinergise.openeo.results/41a32a56-29bd-42d7-9c27-5af98a3421a8/result-0.tiff to /tmp/asdf/result-0.tiff
$ gdalinfo /tmp/results/result-0.tiff
Driver: GTiff/GeoTIFF
...
You can’t perform that action at this time.