Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid errors on zarr dataset settings page #7475

Merged
merged 3 commits into from
Dec 6, 2023
Merged

Conversation

fm3
Copy link
Member

@fm3 fm3 commented Dec 5, 2023

The settings page calls the datastore route GET /datasets/:organizationName/:dataSetName which traverses the files and makes automatic suggestions for the datasource-properties.json. This works only for WKW and was originally designed also as an import step (when dataset creators didn’t write their own datasource-properties.jsons).

This PR skips this step for Zarr and presents a warning instead of a ton of errors (that were there before because the data didn’t match the WKW format).

We can later decide if we want to implement this also for other dataformats like zarr. It could reuse the explore code that is used for remote datasets. However, the suggestions might be imperfect there, as the original datasource-properties.jsons might contain more relevant information that can be inferred from just the zarr array files.

We might also want to discontinue this feature altogether, as zarr becomes more common and datasets are usually either explored remotely (uses the explore code), created via the libs (should already have good datasource-properties.json) or uploaded using the web upload (has its own version of this feature as of #7397). I created follow-up discussion issue #7474 for that

Steps to test:

  • Open the settings page of a zarr dataset
  • Should not show a wall of errors, but rather a single warning

Issues:


  • Updated changelog
  • Needs datastore update after deployment

@fm3 fm3 marked this pull request as ready for review December 5, 2023 09:34
@fm3 fm3 requested a review from frcroth December 5, 2023 10:20
@fm3 fm3 enabled auto-merge (squash) December 6, 2023 08:14
@fm3 fm3 merged commit 513842c into master Dec 6, 2023
2 checks passed
@fm3 fm3 deleted the import-zarr-no-errors branch December 6, 2023 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Settings View for uploaded zarr dataset shows errors
2 participants