Skip to content

Conversation

Vaibhavs10
Copy link
Member

(as the title state - since hf_transfer has no effect w/ xet, let's move to hf_xet)

Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for starting it

accelerate \
numpy==1.26.4 \
hf_transfer \
hf_xet \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the changes in docker limited to these ones:

- diffusers-pytorch-cpu
- diffusers-pytorch-cuda
- diffusers-pytorch-xformers-cuda
- diffusers-pytorch-minimum-cuda
- diffusers-doc-builder

env:
DIFFUSERS_IS_CI: yes
HF_HUB_ENABLE_HF_TRANSFER: 1
HF_XET_HIGH_PERFORMANCE: 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will there be any problems stemming from this in case a repo isn't Xet-transferred? I am guessing this is currently the case i.e., not all repos have gone through this, yet. In that case, would using HF_HUB_ENABLE_HF_TRANSFER have any advantage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question, all repos on the hub are powered by xet. Hence HF_HUB_ENABLE_HF_TRANSFER has no usage whatsoever there

Copy link
Collaborator

@DN6 DN6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍🏽

@sayakpaul
Copy link
Member

I think if we address #12497 (comment), the CI won't cry.

@Vaibhavs10
Copy link
Member Author

I think if we address #12497 (comment), the CI won't cry.

Sorry for not understanding this correctly - but the whole point is to remove all mentions of hf_transfer (it has no effect on xet repos i.e. 100% of the hub) - so I'd rather fix why the CI is not happy with xet in the first place.

More so, the CI errors look more related to model specific integration test that hub - let me know if my understanding is incorrect.

@yiyixuxu
Copy link
Collaborator

cc @DN6
can you take a look? I think this docker-image fail to build pr=12497https://github.com/huggingface/diffusers/actions/runs/18688419900/job/53287370196?pr=12497

@sayakpaul
Copy link
Member

I have pushed a change in 708973b, which should fix the issue. The concerned Docker containers were previously not used in our CI (they are there for bookkeeping and will likely be removed soon).

@sayakpaul sayakpaul merged commit 57636ad into main Oct 21, 2025
14 of 16 checks passed
@sayakpaul
Copy link
Member

Failing tests are unrelated.

@sayakpaul sayakpaul deleted the vb/up-hf-xet branch October 21, 2025 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants