Release 1.4.0.dev20260717001916 · tenstorrent/tt-xla

Installation

Via PyPI

pip install pjrt-plugin-tt==1.4.0.dev20260717001916 --extra-index-url https://pypi.eng.aws.tenstorrent.com/
pip install vllm-tt==1.4.0.dev20260717001916 --extra-index-url https://pypi.eng.aws.tenstorrent.com/

Via Docker

docker pull ghcr.io/tenstorrent/tt-xla-slim:1.4.0.dev20260717001916

What's Changed

[FLUX.1] Add e2e text-to-image pipeline: nightly + benchmark by @ashokkumarkannan1 in #5536
Comparator: use live transformers.Cache so DynamicCache converts after requirements swap by @kamalrajkannan78 in #5644
Lower sharded near-miss PCC thresholds + xfail efficientdet/qwen_3 tests by @ctr-pmuruganTT in #5658
[vLLM] Flip stale opt_level=0 pins to opt_level=1 by @alinakhanTT in #5589
[CI] Run ir processing on weekends (nightly/weekly) by @nsumrakTT in #5662
[vLLM] Relax greedy TP tests to smoke checks on n300_llmbox by @alinakhanTT in #5647

Full Changelog: 1.4.0.dev20260716001804...1.4.0.dev20260717001916

LLM Performance

Model	Token/sec/user	Batch	Token/sec	ttft (ms)	Hardware
pytorch_Falcon_3_10B_Base_nlp_causal_lm_huggingface	41.0	32	1312.0	815.14	p150
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface	57.0	32	1824.0	654.03	n150
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface	108.0	32	3456.0	293.64	p150
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface	37.0	32	1184.0	862.72	n150
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface	70.0	32	2240.0	380.35	p150
pytorch_Falcon_3_7B_Base_nlp_causal_lm_huggingface	19.0	32	608.0	1143.91	n150
pytorch_Falcon_3_7B_Base_nlp_causal_lm_huggingface	37.0	32	1184.0	489.89	p150
pytorch_GLM_4.7_nlp_causal_lm_huggingface	8.0	64	512.0	1611.8	n150
pytorch_GPT-OSS_120B_nlp_causal_lm_huggingface	1.0	128	128.0	37943.3	n150
pytorch_GPT-OSS_120B_nlp_causal_lm_huggingface	4.0	8	32.0	1543.07	p150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface	7.0	64	448.0	6954.72	n150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface	21.0	1	21.0	413.28	p150
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface	40.0	32	1280.0	615.68	n150
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface	81.0	32	2592.0	234.01	p150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface	6.0	32	192.0	8554.85	n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface	22.0	32	704.0	1247.47	n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface	39.0	32	1248.0	573.6	p150
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface	66.0	32	2112.0	586.94	n150
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface	130.0	32	4160.0	226.75	p150
pytorch_Llama_3.2_3B_Instruct_nlp_causal_lm_huggingface	30.0	32	960.0	615.85	n150
pytorch_Llama_3.2_3B_Instruct_nlp_causal_lm_huggingface	55.0	32	1760.0	274.69	p150
pytorch_Mistral_7B_INSTRUCT_v03_nlp_causal_lm_huggingface	35.0	32	1120.0	557.84	p150
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface	12.0	32	384.0	554.39	n150
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface	29.0	32	928.0	222.49	p150
pytorch_Mistral_Small_24B_INSTRUCT_2501_nlp_causal_lm_huggingface	29.0	32	928.0	882.31	p150
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface	21.0	32	672.0	658.26	n150
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface	38.0	32	1216.0	327.56	p150
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface	21.0	32	672.0	655.3	n150
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface	38.0	32	1216.0	329.92	p150
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface	9.0	32	288.0	1244.95	n150
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface	24.0	32	768.0	594.68	p150
pytorch_Qwen 2.5 Coder_32B_Instruct_nlp_causal_lm_huggingface	17.0	32	544.0	1437.36	p150
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface	74.0	32	2368.0	397.5	n150
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface	137.0	32	4384.0	164.45	p150
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface	37.0	32	1184.0	466.24	n150
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface	64.0	32	2048.0	199.55	p150
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface	31.0	32	992.0	712.77	n150
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface	60.0	32	1920.0	273.03	p150
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface	16.0	32	512.0	847.76	n150
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface	27.0	32	864.0	329.69	p150
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface	37.0	32	1184.0	1185.08	n150
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface	75.0	32	2400.0	562.82	p150
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface	28.0	32	896.0	736.59	n150
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface	55.0	32	1760.0	356.29	p150
pytorch_Qwen 3_32B_nlp_causal_lm_huggingface	15.0	32	480.0	1893.69	p150
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface	17.0	32	544.0	1030.97	n150
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface	33.0	32	1056.0	476.34	p150
pytorch_Qwen 3_8B_nlp_causal_lm_huggingface	14.0	32	448.0	1721.87	n150

Non-LLM Performance

Model	Batch	Sample/sec	Hardware
Wan2.2-I2V-A14B-DiT	1	0.0	p150
Wan2.2-I2V-A14B-UMT5-Text-Encoder	1	16.0	p150
Wan2.2-I2V-A14B-VAE-Decoder	1	0.0	p150
Wan2.2-I2V-A14B-VAE-Encoder	1	1.0	p150
flux1-dev	1	0.0	p150
flux2	1	0.0	p150
janus-pro-1b	1	0.0	n150
janus-pro-1b	1	0.0	p150
janus-pro-7b	1	0.0	p150
playground-v2.5	1	0.0	n150
playground-v2.5	1	0.0	p150
pytorch_BERT_emrecan/bert-base-turkish-cased-mean-nli-stsb-tr_nlp_embed_gen_huggingface	8	44.0	n150
pytorch_BERT_emrecan/bert-base-turkish-cased-mean-nli-stsb-tr_nlp_embed_gen_huggingface	8	113.0	p150
pytorch_BGE-M3_Base_nlp_embed_gen_custom	4	9.0	n150
pytorch_BGE-M3_Base_nlp_embed_gen_custom	4	19.0	p150
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm	8	348.0	n150
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm	8	801.0	p150
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom	32	14863.0	n150
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom	32	32908.0	p150
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub	12	1240.0	n150
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub	12	2917.0	p150
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface	32	46.0	n150
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface	32	105.0	p150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface	8	1350.0	n150
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface	8	2757.0	p150
pytorch_Swin_S_cv_image_cls_torchvision	1	10.0	n150
pytorch_Swin_S_cv_image_cls_torchvision	1	22.0	p150
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface	1	4.0	n150
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface	1	8.0	p150
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github	1	136.0	n150
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github	1	238.0	p150
pytorch_VGG19-UNet_base_cv_image_seg_custom	1	152.0	n150
pytorch_VGG19-UNet_base_cv_image_seg_custom	1	306.0	p150
pytorch_ViT_Base_cv_image_cls_huggingface	8	226.0	n150
pytorch_ViT_Base_cv_image_cls_huggingface	8	544.0	p150
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm	8	736.0	n150
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm	8	1559.0	p150
sdxl-lightning	1	0.0	n150
sdxl-lightning	1	0.0	p150

Model coverage

Info: Full list of supported models is available in the assets section.

Model task	Model architecture	Model variant	Model framework	Inference	Training	n150	n300	p150	Single device	Data parallel	Tensor parallel	Model source
conditional generation	U-Net for Conditional Generation	Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	AlexNet	Custom 1x2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
cv image cls	DINOv2	Small	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	EfficientNet	B0	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	MNIST	Cnn Batchnorm	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Dropout	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Dropout	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Nodropout	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom	jax	❌	✅	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom 1x2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
cv image cls	MobileNetV1	Mobilenet v1	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MobileNetV2	Mobilenet v2	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	ResNet	ResNet50 HuggingFace High Resolution	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image cls	SegFormer	Mit B0	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	Swin	S	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	VGG	HF Vgg19	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	ViT	Base	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	VoVNet	Ese Vovnet19b Dw.ra In1k	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image seg	Ultra-Fast Lane Detection	TuSimple ResNet18 Backbone	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image seg	VGG19-UNet	base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv img to img	Autoencoder	linear	pytorch	❌	✅	✅	❌	✅	✅	❌	❌	View Source
cv object det	Attention DenseUNet	Base	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	DETR	ResNet50 Backbone	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	EfficientDet	D0	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	OWL-ViT	Base Patch32	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	PointPillars	pointpillars	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	TransFuser	None	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLO-World	Small 640	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	YOLOP	Default	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLOS Small	Small	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLOv4	Base	pytorch	✅	❌	❌	✅	✅	✅	✅	❌	View Source
cv object det	YOLOv7	Default	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	YOLOv9	T	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	ssd512	ssd512	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm action prediction	OpenVLA-OFT	Finetuned Libero 10	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm image text similarity	CLIP	Base Patch16	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm image text similarity	SigLIP	Base Patch16 224	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm image ttt	FLUX	Transformer	pytorch	✅	❌	❌	❌	❌	✅	❌	❌	Source not available
mm image ttt	FLUX	Vae	pytorch	✅	❌	❌	❌	❌	✅	❌	❌	Source not available
mm visual qa	Mistral	base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	ALLaM	7B Instruct	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	Command_A_Reasoning	command-a-reasoning-08-2025	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Falcon	3 10B Base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Falcon	3 1B Base	pytorch	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Falcon	3 3B Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Falcon	3 7B Base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	GPT-2	Base	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	GPT-2	Xl	jax	❌	✅	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	1.1 2B IT	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	1.1 7B IT	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Gemma	2 27B IT	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Gemma	2 2B IT	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	2 9B IT	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Llama	3.1 70B	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Llama	3.1 70B Instruct	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Llama	3.1 8B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Llama	3.2 1B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Llama	3.2 3B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Llama	3.3 70B Instruct	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	7B INSTRUCT v03	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Mistral	Devstral Small 2505	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Large INSTRUCT 2411	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Magistral Small 2506	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Ministral 8B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Mistral	Nemo INSTRUCT 2407	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Small 24B INSTRUCT 2501	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Phi-1	Phi 1	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Phi-1	Phi 1	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1	Phi 1	pytorch	❌	✅	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1 LoRA	Phi 1	pytorch	❌	✅	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1.5	Phi 1 5	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Phi-1.5	Phi 1 5	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-2	Phi 2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Phi-2	Phi 2	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini 128K Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini 4K Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-4	Phi 4	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Qwen 2	Qwq 32B	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

1.4.0.dev20260717001916

Choose a tag to compare

Sorry, something went wrong.

Sorry, something went wrong.

Uh oh!

No results found

Installation

Via PyPI

Via Docker

What's Changed

What's Changed

LLM Performance

Non-LLM Performance

Model coverage

Contributors

Uh oh!