Skip to content

[fix] missing doc-builder imports#14134

Merged
sayakpaul merged 2 commits into
huggingface:mainfrom
stevhliu:kolors-tokenizer
Jul 8, 2026
Merged

[fix] missing doc-builder imports#14134
sayakpaul merged 2 commits into
huggingface:mainfrom
stevhliu:kolors-tokenizer

Conversation

@stevhliu

@stevhliu stevhliu commented Jul 7, 2026

Copy link
Copy Markdown
Member

#14131 removed the old container which provided sentencepiece which causes the doc builder to break now. This fixes it by guarding the import to fail at use time.

The other thing is to add bitsandbytes to https://github.com/huggingface/doc-builder/blob/main/src/doc_builder/mock_deps/diffusers.txt cc @mishig25

@github-actions github-actions Bot added pipelines size/S PR with diff < 50 LOC labels Jul 7, 2026
@stevhliu stevhliu requested a review from sayakpaul July 7, 2026 17:00
@stevhliu stevhliu changed the title [fix] Guard Kolors tokenizer [fix] missing doc-builder imports Jul 7, 2026
@mishig25

mishig25 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Done in huggingface/doc-builder#805 — adds real:sentencepiece (installed for real by doc-builder light-install; kolors' tokenizer imports it at module level) and bitsandbytes as a mock to mock_deps/diffusers.txt. Validated the full 363-page diffusers docs build in a fresh venv with that registry. Since the shared workflows read the registry from the doc-builder@main checkout, the next main doc build here will pick it up as soon as #805 merges — no diffusers-side change needed for that part. Guarding the import as in this PR is still a good hardening on top.

mishig25 added a commit to huggingface/doc-builder that referenced this pull request Jul 7, 2026
#805)

The old diffusers-doc-builder container provided both; the light path
needs sentencepiece real (kolors' tokenizer imports it at module level)
and bitsandbytes mocked (quantization docs autodoc gated classes).
Requested in huggingface/diffusers#14134.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@mishig25

mishig25 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Update: after #805 (sentencepiece + bitsandbytes) the build progressed and then hit KeyError: 'optimum_quanto' on api/quantization.md. Two findings:

  1. Mock quantization backends (optimum, torchao, gguf) for diffusers docs doc-builder#806 adds the quantization backends (optimum, torchao, gguf) as mocks — with DIFFUSERS_SLOW_IMPORT=yes the quantization page needs them importable, and mocking them means autodoc documents the real config classes rather than dummies. Verified the full 363-page build under exact CI conditions (DIFFUSERS_SLOW_IMPORT=yes, fresh venv, diffusers --no-deps).

  2. The KeyError also exposed a latent diffusers bug the old container was masking: dummy_optimum_quanto_objects.py declares backend "optimum_quanto" but BACKENDS_MAPPING in utils/import_utils.py only has the key "quanto" — so when optimum-quanto is missing, requires_backends crashes with KeyError instead of raising the intended "requires optimum-quanto" ImportError. Worth a one-line fix here (add an "optimum_quanto" key or rename), independent of the doc build.

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@mishig25

mishig25 commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

huggingface/doc-builder#806 is merged and the main doc build now passes ✅ — diffusers is fully on the light path (no container, --no-deps install, sentencepiece real, torch/bitsandbytes/optimum/torchao/gguf mocked), and the build populated the page cache for the runs that follow.

Nothing in the docs pipeline depends on this PR anymore, but the import guard and the optimum_quanto/quanto BACKENDS_MAPPING mismatch noted above are still worth landing on their own merit.

@DN6

DN6 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Thanks for flagging @mishig25 👍🏽 We're planning on removing quanto support, so that should resolve the issue.

@sayakpaul

Copy link
Copy Markdown
Member

Do we still need this change given #14134 (comment)?

@sayakpaul sayakpaul merged commit bb6b33e into huggingface:main Jul 8, 2026
14 of 15 checks passed
@stevhliu stevhliu deleted the kolors-tokenizer branch July 8, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pipelines size/S PR with diff < 50 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants