Skip to content

Latest commit

 

History

History
317 lines (236 loc) · 10.9 KB

release-0.3.md

File metadata and controls

317 lines (236 loc) · 10.9 KB
Raw

EOEPCA System - Release 0.3

Release 0.3 is a beta that includes versions of the following building blocks:

  • Login Service
  • User Profile
  • Policy Enforcement Point (PEP)
  • Policy Decision Point (PDP)
  • Application Deployment & Execution Service (ADES)
  • Processor Development Environment (PDE)
  • Resource Catalogue
  • Data Access Services

Release 0.3 Scope

The release demonstrates the following capabilities:

  • User authentication:
    • Login with GitHub
    • Login with username/password
  • Client Registration
    • Dynamic client registration via SCIM endpoint
  • Authorisation
    • Dynamic Resource Registration
      Resource servers dynamic registration of resources
    • Resource protection
      Enforcing a policy in which resources are owned and protected accordingly
    • Policy-based resource protection
      Enforcing policy based upon policy rules maintained in the PDP
  • Processing Capabilities (ADES resource server)
    • OGC WPS 2.0 and OGC API Processes interfaces
    • List available processes
    • Deploy process (docker container with CWL application package)
    • Execute process (create job)
    • Get job status
    • Data stage-in via OpenSearch catalogue reference
    • Data stage-out to S3 bucket
    • Undeploy process
    • Integration of Calrissian CWL Workflow engine
      Provides native Kubernetes integration and out-of-the-box support for a variety of execution patterns - such fan-in, fan-out, etc.
    • Dedicated user 'context' within ADES service
  • Processor Development Environment
    • JupyterLab interface to interact with code and data
    • Theia IDE to develop using an integrated development environment
    • Local S3 Object Storage with MinIO to store EO data and results
    • Jenkins instance to build the code with Continuous Integration
    • Docker-in-Docker (with an Ubuntu host)
    • Tools for application package testing
  • Resource Catalogue
    • OGC CSW 3.0.0 and 2.0.2 interfaces
    • Certified OGC Compliant and OGC Reference Implementation for both CSW 2.0.2 and CSW 3.0.0
    • Harvesting support for WMS, WFS, WCS, WPS, WAF, CSW, SOS
    • Implements ISO Metadata Application Profile 1.0.0
    • Support for ISO-19115-1 and ISO-19115-2
    • Implements OGC OpenSearch Geo and Time Extensions
    • Implements OGC OpenSearch EO Extensions
    • Federated catalogue distributed searching
  • Data Access Service
    • OGC WMS 1.1 - 1.3 interfaces
    • OGC WMTS 1.0 interfaces with automatic caching
    • OGC WCS 2.0 interfaces with EO Application Profile
    • Implements OGC OpenSearch with EO, Geo and Time Extensions
    • Workspace management API
    • Dataset registration API
    • Registration schemes for Sentinel-2 L1C/L2A data in Data Access Service and Ressource Catalogue
  • End-to-end Processing Execution
    • Authenticated user accessing protected ADES endpoints
    • Dynamic creation of ADES user context with dynamic resource protection
    • Processing inputs discovered in Resource Catalogue
    • Processing inputs accessed via S3 (e.g. CREODIAS eodata)
    • Processing stage-in using STAC file to describe inputs
    • Processing execution on ADES
    • Processing stage-in using STAC file to describe inputs
    • Processing stage-out to S3 bucket
    • Processing stage-out using STAC file to describe outputs
  • Sample application
    Three application packages based-upon s-expression:
    • App s-expression
    • App Water Mask
    • App NVDI

What's new

  • Rule-based policies for resource protection
  • Dynamic registration of protected (per user) ADES resources
  • Dedicated user 'context' within ADES service
  • ADES stage-in/out using STAC manifests
  • ADES stage-out to S3 bucket
  • First version of Resource Catalogue
  • First version of Data Access Components
  • Integration of ADES with Resource Catalogue
  • Helm charts for deployment of all components
  • System deployment using flux continuous delivery kubernetes operator for GitOps

Building Blocks

This section identifies the version of the building blocks components comprising this release, and provides links for further information. For each, we include an 'Example' deployment configuration using a flux HelmRelease resource - these must be adapted for individual deployments.

User Management

Login Service

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Gluu Server:
    • gluufederation/config-init:4.1.1_02
    • gluufederation/wrends:4.1.1_01
    • gluufederation/oxauth:4.1.1_03
    • gluufederation/oxtrust:4.1.1_02

Policy Enforcement Point (PEP)

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • mongo:4.0.23 (latest)

Policy Decision Point (PDP)

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • mongo:4.0.23 (latest)

User Profile

Resources

Resources to support deployment and configuration...

Containers

Processing and Chaining

ADES

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Stage-in: terradue/stars-t2:0.6.17.0
  • Stage-out: terradue/stars-t2:0.6.17.0

Processor Development Environment (PDE)

Sample Application - s-expression

Sample application with 3 application packages for deployment and execution on the ADES.

Resource Management

Resource Catalogue

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Database: postgis/postgis:12-3.1
  • pycsw: geopython/pycsw:2.8.0-alpha1

Data Access Services

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Client: registry.gitlab.eox.at/esa/prism/vs/pvs_client:release-1.1.1
  • Cache: registry.gitlab.eox.at/esa/prism/vs/pvs_cache:release-1.1.1
  • redis: bitnami/redis:6.0.8-debian-10-r0
  • Database: bitnami/postgresql:11.9.0-debian-10-r34