zarr_content_read: include sizes for the "files" (and possibly "directories")

[yarikoptic]

motivation: dandi/dandi-cli#923 (comment) -- in dandi-cli, to possibly avoid relatively expensive checksumming of the content to decide either a (changed) file within zarr needs to be reuploaded, we first check the size(s) of the file. If that one changed -- we know that we need to reupload even without computing its checksum.

It would be helpful if that API endpoint also returned sizes of the files (if it can do it without extra runtime penalty)

[dchiquito]

I am addressing this in #937

[dchiquito]

Reopening this to track the work required to update the dandischema version to 0.6.0, which includes dandi/dandi-schema#120

[jwodder]

Relatedly, if the endpoint also included "last modified" dates and S3 keys (and possibly also S3 version IDs) for Zarr entries, that would help reduce the number of requests needed by the backup script.

[dandibot]

🚀 Issue was released in v0.2.3 🚀

[jwodder]

@dchiquito I don't see how #991 resolves the original issue here. How does one get a file's size from a /zarr/{zarr_id}.zarr/{path} response?

[yarikoptic]

can't test ATM due to #1004 , will reopen just so it doesn't RiP while should be alive and chatty (feel welcome to re-close upon demonstration/confirmation of the effect achieved) ;)

[waxlamp]

@yarikoptic, @jwodder, is this still important? How high of a priority is it at the moment?

[jwodder]

@waxlamp We removed the part of the upload code that checks local vs. remote file sizes (largely because this issue wasn't implemented). On the other hand, the download code still needs to makes a separate HEAD request before each Zarr entry download in order to get the size and modification time.

I'd say it's no longer a priority, but I suppose it would be nice to have.