Skip to content

1.5.0.dev20260814001246

Choose a tag to compare

View all tags
@tenstorrent-github-bot tenstorrent-github-bot released this 14 Aug 00:50
· 2 commits to main since this release
1.5.0.dev20260814001246
4208372
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Installation

Via PyPI

pip install tt-forge==1.5.0.dev20260814001246 --extra-index-url https://pypi.eng.aws.tenstorrent.com/

Via Docker

docker pull ghcr.io/tenstorrent/tt-forge-slim:1.5.0.dev20260814001246

Dependency commits

tt-xla commit: 4e4e88f6859d5f588cdf5ec8e4fbba81c8b868d0
tt-mlir commit: 883ef57adf1ce4c6c4b33deacc5af7d1f09e5e1d
tt-metal commit: 5beed318d0f0d1c6212e605947fb0be80c9e0a1d

What's Changed

  • Uplift third_party/tt_forge_models to 7e486b570e422aa5da81fc10074abd68e7d3b8cb 2026-08-08 by @vvukomanTT in #1024

Full Changelog: 1.5.0.dev20260808000817...1.5.0.dev20260814001246

LLM Performance

Model Token/sec/user Batch Token/sec ttft (ms) Hardware
pytorch_Falcon_3_10B_Base_nlp_causal_lm_huggingface 41.0 32 1312.0 866.2 p150
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface 55.0 32 1760.0 666.65 n150
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface 104.0 32 3328.0 297.58 p150
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface 36.0 32 1152.0 853.47 n150
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface 66.0 32 2112.0 378.01 p150
pytorch_Falcon_3_7B_Base_nlp_causal_lm_huggingface 18.0 32 576.0 1209.67 n150
pytorch_Falcon_3_7B_Base_nlp_causal_lm_huggingface 35.0 32 1120.0 501.78 p150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface 8.0 64 512.0 6246.33 n150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface 21.0 1 21.0 296.12 p150
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface 38.0 32 1216.0 644.36 n150
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface 76.0 32 2432.0 240.67 p150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface 4.0 32 128.0 11575.85 n150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface 12.0 32 384.0 1800.47 p150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface 21.0 32 672.0 1283.41 n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface 38.0 32 1216.0 591.9 p150
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface 64.0 32 2048.0 563.49 n150
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface 121.0 32 3872.0 232.29 p150
pytorch_Llama_3.2_3B_Instruct_nlp_causal_lm_huggingface 30.0 32 960.0 650.7 n150
pytorch_Llama_3.2_3B_Instruct_nlp_causal_lm_huggingface 55.0 32 1760.0 282.26 p150
pytorch_Mistral_7B_INSTRUCT_v03_nlp_causal_lm_huggingface 20.0 32 640.0 1314.36 n150
pytorch_Mistral_7B_INSTRUCT_v03_nlp_causal_lm_huggingface 35.0 32 1120.0 574.08 p150
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface 15.0 32 480.0 576.55 n150
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface 31.0 32 992.0 221.0 p150
pytorch_Mistral_Small_24B_INSTRUCT_2501_nlp_causal_lm_huggingface 19.0 32 608.0 969.14 p150
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface 20.0 32 640.0 664.61 n150
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface 37.0 32 1184.0 318.92 p150
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface 20.0 32 640.0 668.48 n150
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface 37.0 32 1184.0 318.3 p150
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface 8.0 32 256.0 1455.53 n150
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface 20.0 32 640.0 682.12 p150
pytorch_Qwen 2.5 Coder_32B_Instruct_nlp_causal_lm_huggingface 18.0 32 576.0 1484.59 p150
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface 69.0 32 2208.0 410.84 n150
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface 128.0 32 4096.0 170.4 p150
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface 34.0 32 1088.0 507.89 n150
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface 62.0 32 1984.0 204.43 p150
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface 30.0 32 960.0 687.24 n150
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface 57.0 32 1824.0 281.87 p150
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface 16.0 32 512.0 817.56 n150
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface 28.0 32 896.0 353.8 p150
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface 50.0 32 1600.0 1140.77 n150
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface 96.0 32 3072.0 542.27 p150
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface 35.0 32 1120.0 732.49 n150
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface 65.0 32 2080.0 336.02 p150
pytorch_Qwen 3_32B_nlp_causal_lm_huggingface 17.0 32 544.0 1833.88 p150
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface 22.0 32 704.0 1070.9 n150
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface 42.0 32 1344.0 447.03 p150
pytorch_Qwen 3_8B_nlp_causal_lm_huggingface 17.0 32 544.0 1598.72 n150
pytorch_Qwen 3_8B_nlp_causal_lm_huggingface 31.0 32 992.0 734.8 p150

Non-LLM Performance

Model Batch Sample/sec Hardware
Wan2.2-I2V-A14B-DiT 1 0.0 p150
Wan2.2-I2V-A14B-UMT5-Text-Encoder 1 12.0 p150
Wan2.2-I2V-A14B-VAE-Decoder 1 0.0 p150
Wan2.2-I2V-A14B-VAE-Encoder 1 1.0 p150
flux1-dev 1 0.0 p150
flux2 1 0.0 p150
glm-image 1 0.0 p150
hunyuan-image-2.1 1 0.0 p150
janus-pro-1b 1 0.0 n150
janus-pro-1b 1 0.0 p150
janus-pro-7b 1 0.0 p150
playground-v2.5 1 0.0 n150
playground-v2.5 1 0.0 p150
pytorch_BERT_emrecan/bert-base-turkish-cased-mean-nli-stsb-tr_nlp_embed_gen_huggingface 8 158.0 n150
pytorch_BGE-M3_Base_nlp_embed_gen_custom 4 8.0 n150
pytorch_BGE-M3_Base_nlp_embed_gen_custom 4 17.0 p150
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm 8 346.0 n150
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm 8 797.0 p150
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom 32 14396.0 n150
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom 32 30590.0 p150
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub 12 1170.0 n150
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub 12 2792.0 p150
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface 32 46.0 n150
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface 32 96.0 p150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface 8 1309.0 n150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface 8 2766.0 p150
pytorch_SegFormer_B0_Finetuned_Ade_512_512_cv_image_seg_huggingface 1 37.0 n150
pytorch_SegFormer_B0_Finetuned_Ade_512_512_cv_image_seg_huggingface 1 82.0 p150
pytorch_Swin_S_cv_image_cls_torchvision 1 10.0 n150
pytorch_Swin_S_cv_image_cls_torchvision 1 22.0 p150
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface 1 5.0 n150
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface 1 9.0 p150
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github 1 135.0 n150
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github 1 235.0 p150
pytorch_VGG19-UNet_base_cv_image_seg_custom 1 155.0 n150
pytorch_VGG19-UNet_base_cv_image_seg_custom 1 310.0 p150
pytorch_ViT_Base_cv_image_cls_huggingface 8 234.0 n150
pytorch_ViT_Base_cv_image_cls_huggingface 8 563.0 p150
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm 8 744.0 n150
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm 8 1551.0 p150
sdxl-lightning 1 0.0 n150
sdxl-lightning 1 0.0 p150
zimage 1 0.0 p150

Model coverage

Info: Full list of supported models is available in the assets section.

Model task Model architecture Model variant Model framework Inference Training n150 n300 p150 Single device Data parallel Tensor parallel Model source
nlp causal lm Phi-1.5 Phi 1 5 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Cnn Nodropout pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv img to img Autoencoder linear pytorch ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Mistral Nemo INSTRUCT 2407 pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 14B Instruct pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 32B Instruct pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Llama 3.1 70B pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
mm image ttt FLUX Vae pytorch ✅ ❌ ❌ ❌ ❌ ✅ ❌ ❌ Source not available
mm tts xtts_v2 conditioning pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Cnn Dropout pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2 Qwq 32B pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
cv image cls VoVNet Ese Vovnet19b Dw.ra In1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv object det EfficientDet D0 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MobileNetV2 Mobilenet v2 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
mm image text similarity SigLIP Base Patch16 224 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm tts xtts_v2 gpt latents pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 1.5B Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Gemma 2 9B IT pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 3B Instruct jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Phi-2 Phi 2 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 7B Instruct pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
cv object det ssd512 ssd512 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
conditional generation U-Net for Conditional Generation Base pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-1 Phi 1 pytorch ❌ ✅ ❌ ❌ ✅ ✅ ❌ ❌ View Source
nlp embed gen Qwen 3 Embedding 8B pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp embed gen FLUX T5TextEncoder pytorch ✅ ❌ ❌ ❌ ❌ ✅ ❌ ❌ Source not available
nlp token cls BiLSTM-CRF Default pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Mistral 7B INSTRUCT v03 pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
mm tts xtts_v2 gpt decode pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Cnn Dropout jax ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 3 14B pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Falcon 3 7B Base pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Mistral Devstral Small 2505 pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Phi-1 LoRA Phi 1 pytorch ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 3B Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-3 Mini Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm tts xtts_v2 hifigan decoder pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm image ttt FLUX Transformer pytorch ✅ ❌ ❌ ❌ ❌ ✅ ❌ ❌ Source not available
nlp causal lm Qwen 2.5 Coder 0.5B jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm olmo_3 3 7b think pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
cv object det YOLO-World Small 640 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image seg Ultra-Fast Lane Detection TuSimple ResNet18 Backbone pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 3 1 7B jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Llama 3.3 70B Instruct pytorch ✅ ❌ ❌ ❌ ❌ ❌ ❌ ✅ View Source
cv image cls EfficientNet B0 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm Mistral Small 24B INSTRUCT 2501 pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
cv image cls Swin S pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Llama 3.2 1B pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Phi-1.5 Phi 1 5 jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
cv object det TransFuser None pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Mlp Custom 1x2 jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Phi-2 Phi 2 jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
cv panoptic seg Panoptic Segmentation ResNet50 Backbone 1x COCO pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv object det YOLOP Default pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv object det YOLOv4 Base pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
nlp causal lm Gemma 2 2B IT pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
mm visual qa Llama 3.2 11B Vision pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm olmo_3 3 7b instruct pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 0.5B Instruct pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 1.5B Instruct jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm GPT-2 Xl jax ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MNIST Cnn Batchnorm jax ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Gemma 1.1 7B IT pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Llama LoRA 3.2 1B pytorch ❌ ✅ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv object det DETR ResNet50 Backbone pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 3 8B pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 3.5 35B A3B pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm SOLAR-10_7B 10 7B Instruct v1.0 pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
mm tts xtts_v2 speaker encoder pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Qwen 2.5 0.5B Instruct jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Mistral Magistral Small 2506 pytorch ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Qwen 2.5 72B Instruct pytorch ✅ ❌ ❌ ❌ ❌ ❌ ❌ ✅ View Source
nlp causal lm Qwen 2.5 3B jax ✅ ❌ ❌ ✅ ❌ ❌ ❌ ✅ View Source
nlp causal lm Qwen 3 4B pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Gemma 1.1 2B IT pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image seg MaskFormer Swin-B Swin Base Coco pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls ResNet ResNet50 HuggingFace High Resolution pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
nlp causal lm Llama 3.1 8B Instruct pytorch ✅ ❌ ❌ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Falcon 3 1B Base pytorch ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source
nlp causal lm Qwen 2.5 Coder 1.5B jax ✅ ❌ ✅ ✅ ✅ ✅ ❌ ✅ View Source

Contributors

vvukomanTT
Assets 5
Loading