Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow uploading zarr datasets #7397

Merged
merged 21 commits into from
Nov 8, 2023
Merged

Allow uploading zarr datasets #7397

merged 21 commits into from
Nov 8, 2023

Conversation

frcroth
Copy link
Member

@frcroth frcroth commented Oct 17, 2023

URL of deployed dev instance (used for testing):

  • https://___.webknossos.xyz

Steps to test:

  • Upload zarr/ngff datasets as zip
  • Upload a zip containing multiple zarr layers

TODOs:

  • Upload of multiple layers in the same zip
  • Frontend

Issues:


(Please delete unneeded items, merge only when none are left open)

@frcroth frcroth requested a review from fm3 October 23, 2023 16:25
@frcroth frcroth self-assigned this Oct 23, 2023
@frcroth frcroth changed the title WIP: Allow uploading zarr datasets Allow uploading zarr datasets Oct 23, 2023
@frcroth frcroth marked this pull request as ready for review October 23, 2023 16:25
@frcroth frcroth mentioned this pull request Oct 30, 2023
11 tasks
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool stuff, code looks pretty good to me, and I tested a bunch, and most cases worked well. I added a couple of comments, especially about datasets with an existing datasource-properties.json file

@frcroth frcroth requested a review from fm3 November 6, 2023 17:17
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good :) Please see my question about guessed WKW vs Zarr

case UploadedDataSourceType.EXPLORED => Fox.successful(())
case UploadedDataSourceType.WKW =>
for {
p <- tryExploringMultipleZarrLayers(unpackToDir, dataSourceId)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I reading this correctly that we guessed WKW but we still try to explore Zarr? This feels counterintuitive. Maybe the guessing code could be improved instead?

@frcroth frcroth requested a review from fm3 November 8, 2023 08:51
Copy link
Member

@fm3 fm3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice trick with the maxDepth ;) (could you call it by name, with maxDepth=2 though? That way we may stumble less when reading this later)

Looks like the linter also sees an unused import still

Works for me!

@frcroth frcroth merged commit 00901c0 into master Nov 8, 2023
2 checks passed
@frcroth frcroth deleted the zarr-upload branch November 8, 2023 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow uploading ZARR files
3 participants