Repository navigation
Metalhead v0.10.0
Metalhead 0.10 ports models of the Hugging Face Hub to Flux: the vision models of timm, language models of transformers, text-to-image pipelines of diffusers and text embeddings of sentence-transformers. They load the original safetensors weights by name through create_model, and are checked numerically against PyTorch.
Breaking changes
- Flux 0.16.14, NNlib 0.9.57 and MLUtils 0.4.9 are now the minimum versions.
- The hand-written models of 0.9 (
ResNet(50),ViT(:base), ...) and their lowercase builders,backbone,classifier,Metalhead.loadpretrain!, theMetalhead.Layersmodule andpretrain = trueare deprecated, with a warning once per session, and will be removed in 0.11. The migration guide gives theircreate_modelreplacements.
See the changelog for the full list of changes.
🧔♂️🤖 Assisted by AI
Merged pull requests:
- Bump astral-sh/setup-uv from 10.3.0 to 10.3.3 in the all-actions group (#387) (@dependabot[bot])
- Use NNlib.swiglu for the SwiGLU activations (#388) (@CarloLucibello)
- Compute the MoE experts with NNlib.grouped_mul (#389) (@CarloLucibello)
- Key-value cache for the full-attention language models (#390) (@CarloLucibello)
- Default to timm's checkpoint for bare timm architectures (#391) (@CarloLucibello)
- Add generate, with the generation config of the checkpoint (#392) (@CarloLucibello)
- Publish test data (#393) (@github-actions[bot])
- Declare the public API (#394) (@CarloLucibello)
- Publish test data (#395) (@github-actions[bot])
- Merge sub-config overrides and add num_images_per_prompt (#396) (@CarloLucibello)
- Collapse the docs sidebar sections and move the migration guide under Tutorials (#397) (@CarloLucibello)
- Add an index at the top of each API page, and collapse its docstrings (#398) (@CarloLucibello)
- Fix four checkpoint-loading failures (#399) (@CarloLucibello)
- Warn about config.json keys that a model doesn't read (#400) (@CarloLucibello)
- Fix the numerical behavior and initialization items of the 0.10.0 review (#401) (@CarloLucibello)
- Merge the unreleased changelog into 0.10.0 and trim it (#402) (@CarloLucibello)
Closed issues: