Skip to content

1.5.0.dev20260826000452

Choose a tag to compare

View all tags
@tenstorrent-github-bot tenstorrent-github-bot released this 26 Aug 00:46
· 1 commit to main since this release
1.5.0.dev20260826000452
029f25a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Installation

Via PyPI

pip install tt-forge==1.5.0.dev20260826000452 --extra-index-url https://pypi.eng.aws.tenstorrent.com/

Via Docker

docker pull ghcr.io/tenstorrent/tt-forge-slim:1.5.0.dev20260826000452

Dependency commits

tt-xla commit: c106478f15aece8819ae221505ad2e93b2b0d539
tt-mlir commit: e2b21a721955b2b68540d9e6879a215ce3dcfae5
tt-metal commit: d5f9bc4a43b1287523458f1af6a470c6b2adc180

What's Changed

  • Uplift third_party/tt_forge_models to 3d755214570c2cca8efd6a7e15ec878f34ad87ba 2026-08-22 by @vvukomanTT in #1026

Full Changelog: 1.5.0.dev20260822000509...1.5.0.dev20260826000452

LLM Performance

Model Token/sec/user Batch Token/sec ttft (ms) Hardware
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface 9.0 64 576.0 5692.05 n150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface 22.0 1 22.0 290.33 p150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface 7.0 32 224.0 10008.84 n150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface 12.0 32 384.0 1895.3 p150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface 21.0 32 672.0 1210.84 n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface 53.0 32 1696.0 683.24 p150

Non-LLM Performance

Model Batch Sample/sec Hardware
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface 8 1350.0 n150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface 8 2776.0 p150

Model coverage

Info: Full list of supported models is available in the assets section.

Model task Model architecture Model variant Model framework Inference Training n150 n300 p150 Single device Data parallel Tensor parallel Model source
cv object det Attention DenseUNet Base pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv object det DETR ResNet50 Backbone pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MobileNetV2 Mobilenet v2 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm Qwen 2.5 72B Instruct pytorch ✅ ❌ ❌ ❌ ❌ ❌ ❌ ✅ View Source
cv object det YOLOv4 Base pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm Gemma 1.1 2B IT pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm image text similarity SigLIP Base Patch16 224 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 14B Instruct pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Phi-3 Mini 128K Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv object det YOLOv7 Default pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Mlp Custom jax ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 3B Instruct jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Mistral Devstral Small 2505 pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
cv image cls SegFormer Mit B0 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls ViT Base pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm Mistral Ministral 8B Instruct pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 3 8B pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 Coder 3B jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
cv object det OWL-ViT Base Patch32 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm tts xtts_v2 conditioning pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 Coder 3B Instruct jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm GPT-2 Base jax ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 Coder 0.5B jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Mistral Magistral Small 2506 pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Qwen 3.5 35B A3B pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Qwen 2.5 3B jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm SOLAR-10_7B 10 7B Instruct v1.0 pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Gemma 2 9B IT pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 72B pytorch ✅ ❌ ❌ ❌ ❌ ❌ ❌ ✅ View Source
cv image cls MNIST Cnn Dropout jax ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Mlp Custom 1x2 jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Mistral Nemo INSTRUCT 2407 pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 3 14B pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 1.5B Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 3B Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 0.5B Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-2 Phi 2 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 3 0 6B pytorch ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 0.5B Instruct jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
mm action prediction OpenVLA-OFT Finetuned Libero 10 pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
cv object det ssd512 ssd512 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Mlp Custom jax ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-2 Phi 2 jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Qwen 2.5 1.5B Instruct jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm ALLaM 7B Instruct pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
cv object det YOLOP Default pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Falcon 3 3B Base pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp embed gen Qwen 3 Embedding 4B pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-1.5 Phi 1 5 jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Phi-3 Mini 4K Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm olmo_3 3 32b think pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Llama 3.1 70B Instruct pytorch ✅ ❌ ❌ ❌ ❌ ❌ ❌ ✅ View Source
cv image cls MobileNetV1 Mobilenet v1 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp embed gen FLUX ClipTextEncoder pytorch ✅ ❌ ❌ ❌ ❌ ✅ ❌ ❌ Source not available
cv image cls EfficientNet B0 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm GPT-OSS 20B pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm olmo_3 3 7b instruct pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
cv object det YOLO-World Small 640 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 3 4B jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
mm visual qa Mistral base pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Phi-3 Mini Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm tts xtts_v2 speaker encoder pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 0.5B jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp token cls BiLSTM-CRF Default pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-1.5 Phi 1 5 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm tts xtts_v2 hifigan decoder pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Llama LoRA 3.2 1B pytorch ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 Coder 1.5B Instruct jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp embed gen Qwen 3 Embedding 8B pytorch ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Llama 3.1 8B Instruct pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 3.5 27B pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
cv image cls VoVNet Ese Vovnet19b Dw.ra In1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm Qwen 2.5 Coder 1.5B jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Command_A_Reasoning command-a-reasoning-08-2025 pytorch ✅ ❌ ❌ ❌ ❌ ❌ ❌ ✅ View Source
nlp causal lm Llama 3.2 3B pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image seg MaskFormer Swin-B Swin Base Coco pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-1 Phi 1 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm tts xtts_v2 gpt latents pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-1 Phi 1 jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Falcon 3 7B Base pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source

Contributors

vvukomanTT
Assets 5
Loading