fix(artifacts): Allow adding s3 bucket as reference artifact

[ibindlish]

Fixes

• Fixes WB-2758
• Fixes #NNNN

Description

Allows

artifact.add_reference("s3://<bucket_name>")

If the bucket is empty then an empty artifact sequence (i.e., no versions) is created.

🤖 Generated by Copilot at 2ba269f

This pull request enhances the S3Handler class to support empty keys as bucket artifacts and optimize the bucket operations. It modifies the load_path and store_path functions in s3_handler.py to implement these changes.

Testing

Upload

ENTITY = "ibindlish"
PROJECT = "test-s3"
ARTIFACT = "ref-artifact"

with wandb.init(entity=ENTITY, project=PROJECT) as run:
    artifact = wandb.Artifact(ARTIFACT, type="data")
    artifact.add_reference("s3://dev-test-ibindlish")
    run.log_artifact(artifact)

https://wandb.ai/ibindlish/test-s3/artifacts/data/ref-artifact/v2/overview

Download

with wandb.init() as run:
    artifact = run.use_artifact(f"{ENTITY}/{PROJECT}/{ARTIFACT}:v2", type="data")
    artifact_dir = artifact.download()
    print(artifact_dir)

Checklist

• Include reference to internal ticket "Fixes WB-NNNNN" and/or GitHub issue "Fixes #NNNN" (if applicable)
• Ensure PR title compliance with the conventional commits standards

🤖 Generated by Copilot at 2ba269f

The S3Handler class had a flaw
When dealing with buckets so raw
It could not handle keys
That were empty with ease
But this pull request fixes that draw

[codecov]

Codecov Report

Merging #6346 (342c0cd) into main (d2ba674) will decrease coverage by 0.09%.
The diff coverage is 93.33%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6346      +/-   ##
==========================================
- Coverage   77.24%   77.16%   -0.09%     
==========================================
  Files         387      387              
  Lines       44436    44441       +5     
==========================================
- Hits        34323    34291      -32     
- Misses      10060    10097      +37     
  Partials       53       53              

Flag	Coverage Δ
unittest	80.92% <93.33%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
wandb/sdk/artifacts/storage_handlers/s3_handler.py	83.67% <93.33%> (+0.57%)	⬆️

... and 13 files with indirect coverage changes

[moredatarequired]

If we're in a hurry this can merge as-is, but if we have some more hours I'd like to take this opportunity to clean up the function (which was a little ugly before you got to it)

[szymon-piechowicz-wandb]

If we're in a hurry this can merge as-is, but if we have some more hours I'd like to take this opportunity to clean up the function (which was a little ugly before you got to it)

Let's make this a follow up, I think we are under time pressure here.

[moredatarequired]

Typo, extra objs =