feat(artifacts): Add partial file downloads, via directory prefix

[biaslucas]

Description

• Fixes WB-13672

What does the PR do?
Read: https://wandb.atlassian.net/browse/WB-13672

• We allow users to optionally only download files that match the prefix of an input path prefix.
• If nothing is specified, the behavior is as before, we download all paths in the artifact. This prefix match only applies to OUR W&B path structure, NOT references or download file path structures.
• See the below for examples:

Testing

Download prefix case:

Result:

Basecase (no arg given and we download everything):

Final comments: this method is easier to code but a little off from the 'optimal' way of doing things. Ideally within the graphQL query itself we specific a prefix and in the backend we do the filtering. however for now we will just filter after querying filenames (but importantly BEFORE we actually download the files).

There should be ample communication to clients that this argument is prefix / directory filtering for W&B FILEPATHS, not REFERENCE PATHS! they are not the same

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (494df00) 77.75% compared to head (51b5b4f) 79.63%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6911      +/-   ##
==========================================
+ Coverage   77.75%   79.63%   +1.87%     
==========================================
  Files         452      452              
  Lines       50734    50747      +13     
==========================================
+ Hits        39450    40412     +962     
+ Misses      10989    10040     -949     
  Partials      295      295              

Flag	Coverage Δ
func	50.24% <57.14%> (+0.61%)	⬆️
system	64.33% <57.14%> (+0.02%)	⬆️
unit	58.92% <85.71%> (+4.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
wandb/sdk/artifacts/artifact.py	93.07% <100.00%> (+0.10%)	⬆️

... and 38 files with indirect coverage changes

[ibindlish]

@biaslucas I'm wondering if its worth checking whether the path passed in is a file and not a directory, then should we revert to directly downloading a single file instead of iterating over the entire manifest?

[daviddai-evenup]

I think the argument name is path_prefix and not dir_path_prefix

[ibindlish]

I think the argument name is path_prefix and not dir_path_prefix

You're right! Thank you - and apologies since this was misleading!