Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run EOEPCA externally #332

Closed
wants to merge 1 commit into from
Closed

Run EOEPCA externally #332

wants to merge 1 commit into from

Conversation

m-mohr
Copy link
Member

@m-mohr m-mohr commented Mar 4, 2022

As a demo for the tiger team meetings, very basic for now.

Open question:

  • Authentication (requires a non-interactive authentication workflow on side of EOEPCA)
  • How to transfer data cubes forth and back?
  • Same issues as for UDFs apply (e.g. chunking) or the full result needs to be transferred (likely the latter)
  • ...

Rendered variant for better readability:
image

@m-mohr m-mohr requested a review from aljacob March 4, 2022 12:44
@jdries
Copy link
Contributor

jdries commented Mar 7, 2022

I guess you'll also want to define the interface that your EOEPCA process needs to support to be callable by openEO? EOEPCA processes are fully generic AFAIK, so I guess they need to be more specific if we'd want to call them from openEO.

Note that we have done the other way around: calling openEO from EOEPCA processes is super easy and allows for some nice use cases where the EOEPCA process can do multiple calls to openEO, or advanced pre/post processing.

@m-mohr
Copy link
Member Author

m-mohr commented Mar 7, 2022

Hmm... I thought OGC API - Processes is the interface? Maybe I misunderstood.

We should probably mention that the other way around might be better defined and easier to do.

Context: We were asked by Guenther Landgraf how it could work to call EOEPCA from openEO, so this is more a demo/conceptish thing rather than a final proposal.

"parameters": [
{
"name": "data",
"description": "The data to be passed to the external processing platform.",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implicitly does a save_result to a file format the image can read.

"description": "The data processed by the external processing platform.",
"schema": {
"type": "object",
"subtype": "raster-cube"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the application package would implictly need to save a result in a way that openEO can read it. The allowed file formats are available through GET /file_formats, but there might be issues in the details (e.g. how are dimensions named in netCDF, etc.)

{
"id": "run_eoepca_externally",
"summary": "Run processing externally on EOEPCA",
"description": "Runs processing tasks on an [EOEPCA](https://eoepca.org/) compliant processing platform, which essentially runs a hosted docker container.",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add information about the API interface. Is this always OGC API - Processes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs to be much more specific than just saying OGC API - Processes.
EOEPCA should have a kind of profile with process signatures that an application package can support to be callable by openEO.

@jdries
Copy link
Contributor

jdries commented Mar 11, 2022

Note that from a user perspective, calling an external service may not be necessary. In fact, if there's an application package that satisfies a few conditions, an openEO backend can use it in a way that's very similar to our UDF setup.
This would also avoid having to solve the problem of authentication between these services.

@m-mohr m-mohr closed this Jul 19, 2022
@m-mohr m-mohr deleted the run_eoepca_externally branch July 19, 2022 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants