Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to general data cube definition #382

Merged
merged 15 commits into from
Jan 30, 2023
Merged

Migrate to general data cube definition #382

merged 15 commits into from
Jan 30, 2023

Conversation

m-mohr
Copy link
Member

@m-mohr m-mohr commented Sep 7, 2022

This goes through the existing processes and makes changes so that, if applicable, they work with vector cubes.

Tasks:

  • Replace raster-cube subtype with datacube
  • Define datacube dimension minimum requirements
  • Update tests
  • Update descriptions (e.g. check mentioning of "pixel values" or raster data cubes)
  • Check geojson parameters (and also allow vector-cubes)
  • Add guidelines how geojson should be converted into a vector-cube (as we have geojson as input data type it might be the only file format for which we describe the actual import procedure) -> I'll keep this as a separate PR, tracked by process to convert inline GeoJSON to a vector cube #346
  • Update dimension types, e.g. in add_dimension
  • Update subtype-schemas.json
  • Process specific:
    • Update aggregate_spatial with all its flaws and proposed changes
    • load_collection/load_results: Update descriptions to reflect vector use cases
    • filter_bbox/spatial: Update description to reflect vector use cases (only include values that are fully included in the given extent?)
    • What dimension types are required in ard_normalized_radar_backscatter and sar_backscatter? spatial + bands?
    • merge_cubes needs new/more examples
    • Does it make sense to extend climatological_normal from raster-cubes to general datacubes?

@m-mohr m-mohr linked an issue Oct 9, 2022 that may be closed by this pull request
aggregate_spatial.json Outdated Show resolved Hide resolved
@m-mohr
Copy link
Member Author

m-mohr commented Jan 17, 2023

For another issue: Should we change the vector_ prefix for the vector-related processes to something else?

Edit: Thinking a bit more about it, I think vector_ is fine. We still have a vector data cube as input, only the dimension is using the geometry terminology. As the prefixes do usually reflect the type of data you are passing in (i.e. vector data cube), we can stick with the names. Also to preserve a bit of backward compatibility with some rare implementations.

@soxofaan
Copy link
Member

For another issue: Should we change the vector_ prefix for the vector-related processes to something else?

Also: let's move that question to another issue, to unblock #382

@m-mohr m-mohr linked an issue Jan 18, 2023 that may be closed by this pull request
@m-mohr
Copy link
Member Author

m-mohr commented Jan 18, 2023

This is also ready for final review. I hope that I have fixed all open issues sufficiently. If you see minor things that we can fix quickly, please mention them here. If you see larger issues to be discussed, maybe open a separate issue so that we can merge this and then work in smaller PRs that are easier to handle.

Additionally, the schema for vector data cubes has changed due to the changes in the other PRs (API, STAC etc.):

                {
                    "type": "object",
                    "subtype": "datacube",
                    "dimensions": [
                        {
                            "type": "geometries"
                        }
                    ]
                }

So the type is now geometries instead of vector.

@soxofaan @mkadunc @aljacob @clausmichele @jdries @dthiex @LukeWeidenwalker @pierocampa

@m-mohr
Copy link
Member Author

m-mohr commented Jan 18, 2023

Also: let's move that question to another issue, to unblock #382

See #397

Copy link
Member

@aljacob aljacob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks all good to me. My only concern would also be about the plural of geometries and bands, it would be cleaner to have both in singular, but it is nothing that is a big issue after all, at least these two are consistant then, as I guess it would be quite the effort for everyone to go from bands to band as well.

@m-mohr
Copy link
Member Author

m-mohr commented Jan 30, 2023

@aljacob The naming is discussed in another issue: Open-EO/openeo-api#479

@m-mohr m-mohr merged commit a5764ed into draft-2.0 Jan 30, 2023
@m-mohr m-mohr deleted the datacubes branch January 30, 2023 15:30
@m-mohr m-mohr mentioned this pull request Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants