Skip to content

Latest commit

 

History

History
376 lines (283 loc) · 13.4 KB

release-0.9.md

File metadata and controls

376 lines (283 loc) · 13.4 KB
Raw

EOEPCA System - Release 0.9

Release 0.9 is a beta that includes versions of the following building blocks:

  • Login Service
  • User Profile
  • Policy Enforcement Point (PEP)
  • Policy Decision Point (PDP)
  • Application Deployment & Execution Service (ADES)
  • Processor Development Environment (PDE)
  • Resource Catalogue
  • Data Access Services
  • Workspace

See the Release v0.9 Description for more details.

What's new

  • User Workspace
    Providing managment and access to user owned resources, including full resource catalogue and data access services.
    Workspace resources are access protected to the owning user.
  • Integration of ADES with Protected User Workspace
    Protected access to user's workspace, authorized according to policy.
    User's processing results are pushed to their Workspace bucket, and registered in their Resource Catalogue and Data Access Services.
  • Commercial Operator Identity Hub (COIH)
    Integration of ESA's Commercial Operator Identity Hub (COIH) as an identity provider.
  • OGC API Records
    Support added to Resource Catalogue.
  • EOEPCA Helm Chart Repositories
  • Normalized Hotspot Indices
    New sample application for ADES deployment.

Release 0.9 Scope

The release demonstrates the following capabilities:

  • User authentication:
    • Login with GitHub
    • Login with ESA Commercial Operator Identity Hub (COIH)
    • Login with username/password
  • Client Registration
    • Dynamic client registration via SCIM endpoint
  • Authorisation
    • Dynamic Resource Registration
      Resource servers dynamic registration of resources
    • Resource protection
      Enforcing a policy in which resources are owned and protected accordingly
    • Policy-based resource protection
      Enforcing policy based upon policy rules maintained in the PDP
  • Processing Capabilities (ADES resource server)
    • OGC WPS 2.0 and OGC API Processes interfaces
    • Secure protected resource server, with access policy enforcement via PEP
    • List available processes
    • Deploy process (docker container with CWL application package)
    • Execute process (create job)
    • Get job status
    • Data stage-in via OpenSearch catalogue reference
    • Data stage-out to S3 bucket
    • Undeploy process
    • Integration of Calrissian CWL Workflow engine
      Provides native Kubernetes integration and out-of-the-box support for a variety of execution patterns - such fan-in, fan-out, etc.
    • Dedicated user 'context' within ADES service
  • Processor Development Environment
    • JupyterLab interface to interact with code and data
    • Theia IDE to develop using an integrated development environment
    • Local S3 Object Storage with MinIO to store EO data and results
    • Jenkins instance to build the code with Continuous Integration
    • Docker-in-Docker (with an Ubuntu host)
    • Tools for application package testing
  • Resource Catalogue
    • OGC CSW 3.0.0 and 2.0.2 interfaces
      • Certified OGC Compliant and OGC Reference Implementation for both CSW 2.0.2 and CSW 3.0.0
      • Harvesting support for WMS, WFS, WCS, WPS, WAF, CSW, SOS
      • Federated catalogue distributed searching
    • Implements ISO Metadata Application Profile 1.0.0
    • Support for ISO-19115-1 and ISO-19115-2
    • OpenSearch
      • OGC OpenSearch Geo and Time Extensions
      • OGC OpenSearch EO Extensions
    • OGC API Records
  • Data Access Service
    • OGC WMS 1.1 - 1.3 interfaces
    • OGC WMTS 1.0 interfaces with automatic caching
    • OGC WCS 2.0 interfaces with EO Application Profile
    • OGC OpenSearch with EO, Geo and Time Extensions
    • Workspace management API
    • Dataset registration API
    • Registration schemes for Sentinel-2 L1C/L2A data in Data Access Service and Ressource Catalogue
  • End-to-end Processing Execution
    • Authenticated user accessing protected ADES endpoints
    • Dynamic creation of ADES user context with dynamic resource protection
    • Processing inputs discovered in Resource Catalogue
    • Processing inputs accessed via S3 (e.g. CREODIAS eodata)
    • Processing stage-in using STAC file to describe inputs
    • Processing execution on ADES
    • Processing stage-in using STAC file to describe inputs
    • Processing stage-out to S3 bucket
    • Processing stage-out using STAC file to describe outputs
    • Secure interfacing between ADES and user's protected Workspace
      ADES client -> Get Workspace Details / (De-)Register Resources
  • User Workspace
    • Secure protected management interface (workspace-api), with access policy enforcement via PEP
    • Management functions via REST API...
      • Create workspace (admin)
      • Get workspace details (user)
      • Delete workspace (admin)
      • Patch workspace (admin)
      • Redeploy workspace (admin)
      • Register resources (user)
      • Deregister resources (user)
    • User-specific S3 bucket for resource storage
    • User-specific Resource Catalogue
    • User-specific Data Access Services
  • Sample application: s-expression for EO product band math
    Three application packages based-upon s-expression:
    • App s-expression
    • App Water Mask
    • App NVDI
  • Sample application: Normalized Hotspot Indices

Building Blocks

This section identifies the version of the building blocks components comprising this release, and provides links for further information. For each, we include an 'Example' deployment configuration using a flux HelmRelease resource - these must be adapted for individual deployments.

User Management

Login Service

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Gluu Server:
    • gluufederation/config-init:4.1.1_02
    • gluufederation/oxauth:4.1.1_03
    • gluufederation/oxtrust:4.1.1_02
    • gluufederation/wrends:4.1.1_01

Policy Enforcement Point (PEP)

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • mongo:5.0.0 (latest)

Policy Decision Point (PDP)

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • mongo:5.0.0 (latest)

User Profile

Resources

Resources to support deployment and configuration...

Containers

Processing and Chaining

ADES

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Stage-in: terradue/stars-t2:0.6.18.19
  • Stage-out: terradue/stars-t2:0.6.18.19

Processor Development Environment (PDE)

Sample Application - s-expression for EO product band math

Sample application with 3 application packages for deployment and execution on the ADES.

Sample Application - Normalized Hotspot Indices

Sample application...

Resource Management

Resource Catalogue

Resources

Resources to support deployment and configuration...

Containers

  • pycsw (version 0.9.0)
    • Image: geopython/pycsw:eoepca-0.9.0

Additional container images:

  • Database: postgis/postgis:12-3.1

Data Access Services

Resources

Resources to support deployment and configuration...

Containers

Additional container images:

  • Client: registry.gitlab.eox.at/esa/prism/vs/pvs_client:release-1.3.8
  • Cache: registry.gitlab.eox.at/esa/prism/vs/pvs_cache:release-1.3.8
  • redis: bitnami/redis:6.0.8-debian-10-r0
  • Database: bitnami/postgresql:11.9.0-debian-10-r34

Workspace

Resources

Resources to support deployment and configuration...

Containers