Skip to content

Feature/8754 gridded products variables across zarr stores - #317

Merged
utas-raymondng merged 3 commits into
mainfrom
feature/8754-gridded-products-variables-across-zarr-stores
Aug 11, 2026
Merged

Feature/8754 gridded products variables across zarr stores#317
utas-raymondng merged 3 commits into
mainfrom
feature/8754-gridded-products-variables-across-zarr-stores

Conversation

@weited

@weited weited commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@weited
weited marked this pull request as draft July 31, 2026 04:57
@weited
weited requested a review from Copilot July 31, 2026 04:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a local, startup-validated DAS tile-product catalogue (YAML + registry) so tile product discovery and route membership checks no longer depend on upstream DAS availability, while keeping per-date availability dynamic via a dedicated proxy endpoint.

Changes:

  • Add das-tile-products.yaml as the source of truth for which tile products exist, plus DasTileProduct, DasTileProductsProperties, and DasTileProductRegistry with fail-fast startup validation.
  • Update tile APIs to use the local registry for product listing and membership checks; split per-date availability into GET .../available_dates and add a catalogue-wide GET .../products.
  • Refactor and expand tests to use a deterministic test-profile catalogue and to validate the production seed catalogue binds and passes registry validation.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
server/src/test/resources/application-test.yaml Adds a small deterministic das-tile-products catalogue for tests.
server/src/test/java/au/org/aodn/ogcapi/server/tile/RestExtApiTest.java Updates/expands ext tiles tests for local catalogue listings and new availability endpoint behavior.
server/src/test/java/au/org/aodn/ogcapi/server/tile/RestApiTest.java Updates visual tile route tests to use real UUIDs and local-catalogue validation rules.
server/src/test/java/au/org/aodn/ogcapi/server/core/service/das/DasTilerServiceTest.java Updates DAS client tests to cover the new getManifest(from,to) behavior and error mapping.
server/src/test/java/au/org/aodn/ogcapi/server/core/service/das/DasTileProductTest.java Adds unit tests for DasTileProduct defaulting and id derivation.
server/src/test/java/au/org/aodn/ogcapi/server/core/service/das/DasTileProductsSeedTest.java Adds a guard-rail test that binds and validates the production seed catalogue.
server/src/test/java/au/org/aodn/ogcapi/server/core/service/das/DasTileProductRegistryTest.java Adds unit tests for registry indexing, lookup, membership validation, and startup rules.
server/src/main/resources/das-tile-products.yaml Introduces the production tile-product catalogue (hand-maintained seed).
server/src/main/resources/application.yaml Imports the new catalogue YAML via spring.config.import.
server/src/main/java/au/org/aodn/ogcapi/server/tile/RestExtApi.java Switches product listing to local catalogue, adds global /products, adds /available_dates, and updates data tile routes to use registry validation.
server/src/main/java/au/org/aodn/ogcapi/server/tile/RestApi.java Updates visual tile route to validate membership via the local registry and enforce canVisual().
server/src/main/java/au/org/aodn/ogcapi/server/core/service/das/DasTilerService.java Removes upstream products listing; updates manifest fetching to support optional from/to bounds.
server/src/main/java/au/org/aodn/ogcapi/server/core/service/das/DasTileProductRegistry.java Adds the in-memory index and exhaustive startup validation for the catalogue.
server/src/main/java/au/org/aodn/ogcapi/server/core/service/das/DasTileProduct.java Adds the catalogue entry record with derived id and defaulted tile_types.
server/src/main/java/au/org/aodn/ogcapi/server/core/configuration/DasTileProductsProperties.java Adds Spring @ConfigurationProperties binding for the catalogue namespace.
server/src/main/java/au/org/aodn/ogcapi/server/core/configuration/Config.java Registers DasTileProductsProperties for config binding.
Suppressed comments (1)

server/src/main/java/au/org/aodn/ogcapi/server/tile/RestExtApi.java:520

  • getCollectionDataManifest also proxies upstream without checking the product’s tile_types. For consistency with the product listing (which conditionally emits data_manifest_url_template) and with the visual route’s capability check, reject products that don’t include DATA before calling DAS.
        // SEAM: local-config check only. If catalogue typos / dataset renames become a problem,
        // re-add dasTilerService.isDatasetInCollection(collectionId, dataset) here as a second,
        // ES-backed cross-check (see the dormant method's javadoc in DasTilerService).
        DasTileProduct product = dasTileProductRegistry.validateForCollection(collectionId, dataset, variable);

        DasTilerService.DasJsonResult manifest = dasTilerService.getDataManifest(product.id(), datetime);


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +189 to +191
ArrayNode tileTypes = mapper.createArrayNode();
product.tileTypes().forEach(type -> tileTypes.add(type.name().toLowerCase()));
entry.set("tile_types", tileTypes);
}

/** The variable half of the product id: lowercased and {@code +}-joined for a vector. */
public String variablePart() { return String.join("+", variables).toLowerCase(); }
Comment on lines +433 to +438
// SEAM: local-config check only. If catalogue typos / dataset renames become a problem,
// re-add dasTilerService.isDatasetInCollection(collectionId, dataset) here as a second,
// ES-backed cross-check (see the dormant method's javadoc in DasTilerService).
DasTileProduct product = dasTileProductRegistry.validateForCollection(collectionId, dataset, variable);

// DAS identifies a product by the combined {dataset}:{variable} id.
String product = dataset + ":" + variable;
DasTilerService.DasTileResult tile = dasTilerService.getDataTile(product, datetime, lod, x, y);
DasTilerService.DasTileResult tile = dasTilerService.getDataTile(product.id(), datetime, lod, x, y);
@weited weited closed this Aug 5, 2026
@weited
weited force-pushed the feature/8754-gridded-products-variables-across-zarr-stores branch from 1fc9c50 to 45298fb Compare August 5, 2026 05:18
@weited weited reopened this Aug 9, 2026
@weited
weited marked this pull request as ready for review August 10, 2026 00:07
@weited
weited force-pushed the feature/8754-gridded-products-variables-across-zarr-stores branch from e55143b to 2841289 Compare August 10, 2026 00:07
@weited
weited requested a review from utas-raymondng August 10, 2026 00:13
@weited
weited force-pushed the feature/8754-gridded-products-variables-across-zarr-stores branch from 2841289 to 4b7ef78 Compare August 10, 2026 00:38

@utas-raymondng utas-raymondng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@utas-raymondng
utas-raymondng merged commit fbb3d50 into main Aug 11, 2026
4 checks passed
@utas-raymondng
utas-raymondng deleted the feature/8754-gridded-products-variables-across-zarr-stores branch August 11, 2026 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants