Skip to content

1.4.0.dev20260715001607

Choose a tag to compare

View all tags
@github-actions github-actions released this 15 Jul 02:32
1.4.0.dev20260715001607
279b5d9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Installation

Via PyPI

pip install pjrt-plugin-tt==1.4.0.dev20260715001607 --extra-index-url https://pypi.eng.aws.tenstorrent.com/
pip install vllm-tt==1.4.0.dev20260715001607 --extra-index-url https://pypi.eng.aws.tenstorrent.com/

Via Docker

docker pull ghcr.io/tenstorrent/tt-xla-slim:1.4.0.dev20260715001607

What's Changed

What's Changed

Full Changelog: 1.4.0.dev20260714020907...1.4.0.dev20260715001607

LLM Performance

Model Token/sec/user Batch Token/sec ttft (ms) Hardware
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface 57.0 32 1824.0 691.02 n150
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface 107.0 32 3424.0 331.4 p150
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface 37.0 32 1184.0 899.67 n150
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface 70.0 32 2240.0 378.1 p150
pytorch_Falcon_3_7B_Base_nlp_causal_lm_huggingface 19.0 32 608.0 1152.82 n150
pytorch_Falcon_3_7B_Base_nlp_causal_lm_huggingface 36.0 32 1152.0 497.87 p150
pytorch_GLM_4.7_nlp_causal_lm_huggingface 8.0 64 512.0 1623.36 n150
pytorch_GPT-OSS_120B_nlp_causal_lm_huggingface 1.0 128 128.0 39157.16 n150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface 7.0 64 448.0 6740.2 n150
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface 40.0 32 1280.0 620.58 n150
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface 81.0 32 2592.0 232.45 p150
pytorch_Kimi-K2.6_kimi_k2_6_modified_nlp_causal_lm_custom 5.0 64 320.0 3891.42 n150
pytorch_Kimi-K2_kimi_k2_instruct_modified_nlp_causal_lm_custom 5.0 64 320.0 3901.59 n150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface 6.0 32 192.0 8731.03 n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface 22.0 32 704.0 1248.65 n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface 38.0 32 1216.0 574.27 p150
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface 66.0 32 2112.0 570.87 n150
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface 129.0 32 4128.0 224.08 p150
pytorch_Llama_3.2_3B_Instruct_nlp_causal_lm_huggingface 31.0 32 992.0 582.58 n150
pytorch_Llama_3.2_3B_Instruct_nlp_causal_lm_huggingface 57.0 32 1824.0 276.41 p150
pytorch_Mistral_7B_INSTRUCT_v03_nlp_causal_lm_huggingface 20.0 32 640.0 1207.29 n150
pytorch_Mistral_7B_INSTRUCT_v03_nlp_causal_lm_huggingface 36.0 32 1152.0 560.09 p150
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface 13.0 32 416.0 556.74 n150
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface 29.0 32 928.0 222.19 p150
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface 21.0 32 672.0 676.75 n150
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface 39.0 32 1248.0 321.78 p150
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface 21.0 32 672.0 652.3 n150
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface 38.0 32 1216.0 325.0 p150
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface 9.0 32 288.0 1237.0 n150
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface 24.0 32 768.0 594.62 p150
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface 69.0 32 2208.0 405.97 n150
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface 135.0 32 4320.0 168.45 p150
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface 37.0 32 1184.0 463.88 n150
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface 63.0 32 2016.0 198.17 p150
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface 31.0 32 992.0 676.23 n150
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface 60.0 32 1920.0 271.24 p150
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface 16.0 32 512.0 846.31 n150
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface 28.0 32 896.0 336.33 p150
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface 36.0 32 1152.0 1222.76 n150
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface 75.0 32 2400.0 557.11 p150
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface 29.0 32 928.0 746.44 n150
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface 56.0 32 1792.0 356.31 p150
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface 17.0 32 544.0 1031.15 n150
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface 33.0 32 1056.0 474.92 p150
pytorch_Qwen 3_8B_nlp_causal_lm_huggingface 13.0 32 416.0 1629.72 n150
pytorch_Qwen 3_8B_nlp_causal_lm_huggingface 25.0 32 800.0 757.07 p150

Non-LLM Performance

Model Batch Sample/sec Hardware
janus-pro-1b 1 0.0 n150
janus-pro-1b 1 0.0 p150
janus-pro-7b 1 0.0 p150
playground-v2.5 1 0.0 n150
playground-v2.5 1 0.0 p150
pytorch_BERT_emrecan/bert-base-turkish-cased-mean-nli-stsb-tr_nlp_embed_gen_huggingface 8 44.0 n150
pytorch_BERT_emrecan/bert-base-turkish-cased-mean-nli-stsb-tr_nlp_embed_gen_huggingface 8 113.0 p150
pytorch_BGE-M3_Base_nlp_embed_gen_custom 4 9.0 n150
pytorch_BGE-M3_Base_nlp_embed_gen_custom 4 19.0 p150
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm 8 348.0 n150
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm 8 792.0 p150
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom 32 15127.0 n150
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom 32 32887.0 p150
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub 12 1239.0 n150
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub 12 2915.0 p150
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface 32 47.0 n150
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface 32 105.0 p150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface 8 1353.0 n150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface 8 2764.0 p150
pytorch_Swin_S_cv_image_cls_torchvision 1 10.0 n150
pytorch_Swin_S_cv_image_cls_torchvision 1 22.0 p150
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface 1 4.0 n150
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface 1 8.0 p150
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github 1 135.0 n150
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github 1 238.0 p150
pytorch_VGG19-UNet_base_cv_image_seg_custom 1 154.0 n150
pytorch_VGG19-UNet_base_cv_image_seg_custom 1 307.0 p150
pytorch_ViT_Base_cv_image_cls_huggingface 8 226.0 n150
pytorch_ViT_Base_cv_image_cls_huggingface 8 545.0 p150
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm 8 733.0 n150
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm 8 1560.0 p150
sdxl-lightning 1 0.0 n150
sdxl-lightning 1 0.0 p150

Model coverage

Info: Full list of supported models is available in the assets section.

Model task Model architecture Model variant Model framework Inference Training n150 n300 p150 Single device Data parallel Tensor parallel Model source
atomic ml ARNOLD Deathmatch Shotgun Rnn pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
atomic ml ARNOLD Defend The Center Ff pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
atomic ml ARNOLD Health Gathering Ff pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
atomic ml ARNOLD Vizdoom 2017 Track1 Rnn pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
atomic ml ARNOLD Vizdoom 2017 Track2 Rnn pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv depth est GLPN-KITTI base pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv depth est MonoDepth2 Mono 1024x320 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Mono 640x192 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Mono No Pt 640x192 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Mono+stereo 1024x320 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Mono+stereo 640x192 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Mono+stereo No Pt 640x192 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Stereo 1024x320 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Stereo 640x192 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv depth est MonoDepth2 Stereo No Pt 640x192 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls AlexNet B pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls AlexNet Default pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls AlexNet High Res pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls BEiT Base pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls BEiT Large pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DINOv2 Base pytorch ✅ ❌ ✅ ❌ ❌ ✅ ❌ ❌ View Source
cv image cls DINOv2 Giant pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls DINOv2 Large pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls DLA 102 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 102x pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 102x2 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 169 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 34 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 46 C pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 46x C pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 60 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 60x pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DLA 60x C pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls DeiT Base pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls DeiT Base Distilled pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DeiT Small pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DeiT Tiny pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DenseNet 121 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DenseNet 121 Xray pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls DenseNet 161 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DenseNet 169 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls DenseNet 201 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet B1 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet B2 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet B3 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet B4 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet B5 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet B6 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet B7 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet Timm B0 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet Timm B0 Ra In1k pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet Timm B4 pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet Timm B4 Ra2 In1k pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet Timm B5 In12k Ft In1k pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet Timm Tf B0 Aa In1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet Timm Tf V2 S In21k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet Timm V2 Rw S Ra2 In1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet-Lite Tf Efficientnet Lite0.in1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet-Lite Tf Efficientnet Lite1.in1k pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls EfficientNet-Lite Tf Efficientnet Lite2.in1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet-Lite Tf Efficientnet Lite3.in1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls EfficientNet-Lite Tf Efficientnet Lite4.in1k pytorch ✅ ❌ ✅ ✅ ❌ ✅ ✅ ❌ View Source
cv image cls FPN ResNet50 Backbone with FPN V2 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls GhostNet 100 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls GhostNet 100.in1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls GhostNet v2 100.in1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls GoogLeNet Default pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls HarDNet hardnet68 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls Inception V4.tf In1k pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls Inception v4 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls Inception v4 OSMR pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MLP-Mixer Base pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MLP-Mixer Mixer B16 224 pytorch ✅ ❌ ✅ ✅ ❌ ✅ ✅ ❌ View Source
cv image cls MLP-Mixer Mixer B16 224 In21k pytorch ✅ ❌ ✅ ✅ ❌ ✅ ✅ ❌ View Source
cv image cls MLP-Mixer Mixer B16 224 Miil pytorch ✅ ❌ ✅ ✅ ❌ ✅ ✅ ❌ View Source
cv image cls MLP-Mixer Mixer B16 224 Miil In21k pytorch ✅ ❌ ✅ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MLP-Mixer Mixer B32 224 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls MLP-Mixer Mixer Github pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source
cv image cls MLP-Mixer Mixer L16 224 In21k pytorch ✅ ❌ ❌ ❌ ✅ ✅ ❌ ❌ View Source
cv image cls MLP-Mixer Mixer L32 224 pytorch ✅ ❌ ✅ ✅ ✅ ✅ ✅ ❌ View Source

Contributors

mmanzoorTT, vvukomanTT, and 2 other contributors
Assets 5
Loading