Skip to content

Filter HuggingFace downloads by target model and adapter name (fixes #3) - #25

Merged
antonpibm merged 3 commits into
mainfrom
feature/filter_downloaded_artifacts
May 13, 2026
Merged

Filter HuggingFace downloads by target model and adapter name (fixes #3)#25
antonpibm merged 3 commits into
mainfrom
feature/filter_downloaded_artifacts

Conversation

@NoaTal1996

Copy link
Copy Markdown
Collaborator

Fixes #3. Adapter libraries used to be downloaded in full before any
filter was applied. This PR builds allow_patterns from --base-model,
--include-adapters, and --exclude-adapters before calling
snapshot_download, so only the requested paths are fetched.

Verification

Full compose produces a BUILD.md with the same 12 adapters and param
counts as the published ibm-granite/granite-switch-4.1-3b-preview.

Tests

Added tests/composer/test_selective_download.py — 32 tests covering the
helpers, the allow_patterns construction, a verbatim repro of issue #3,
and real Hub metadata calls. Passes locally and on Vela1.

@antonpibm
antonpibm merged commit 241d549 into main May 13, 2026
@antonpibm
antonpibm deleted the feature/filter_downloaded_artifacts branch May 13, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--include-adapters and --base-model filters are applied after download — all adapter sizes and names are fetched unnecessarily

2 participants