1.2.0

Installation

Via PyPI

pip install tt-forge==1.2.0 --extra-index-url https://pypi.eng.aws.tenstorrent.com/

Via Docker

docker pull ghcr.io/tenstorrent/tt-forge-slim:1.2.0

What's Changed

• Add workflow to test installations steps using AI by @vmilosevic in #921
• Uplift third_party/tt_forge_models to 37332bc2fa41c6d8ff7ec994ad19b84ab266429f 2026-03-28 by @vvukomanTT in #922
• Improve claude instructions testing by @vmilosevic in #923
• Bump release version to 1.1.0 by @vvukomanTT in #924
• Add a number of skills for the tenstorrent ecosystem by @zoecarver in #925
• Update claude workflow by @vmilosevic in #926
• Uplift third_party/tt_forge_models to e7ba96f22de7ced01b151393f4d4e1bc312fac23 2026-04-04 by @vvukomanTT in #930
• Remove benchmark infrastructure from tt-forge by @chandrasekaranpradeep in #928
• Upgrade checkout from v4 to v5, Add venv activate by @nsumrakTT in #929
• Uplift third_party/tt_forge_models to 752832c839d59bb6c3fde43d7b9c03ebc820fa57 2026-04-11 by @vvukomanTT in #931
• Add tt-xla llama demo by @ddilbazTT in #932
• Add tiny llama demo by @ddilbazTT in #933
• Uplift third_party/tt_forge_models to 534f4821827f4e92af6a1ad58e8c836c8ccd38fd 2026-04-18 by @vvukomanTT in #935
• Stop tt-forge-onnx nightly releases by @vvukomanTT in #936
• Uplift third_party/tt_forge_models to d11fe6671e44d286c2e8c6dcd9bd46a3b84f9ea8 2026-04-25 by @vvukomanTT in #937
• Uplift third_party/tt_forge_models to cae9ccbc67a318736f656bee9a9ea776eb73e69c 2026-05-02 by @vvukomanTT in #940
• Remove depreciated files by @nsumrakTT in #941
• Add qwen3 demo by @abrown in #934
• Add filter test matrix file by @vvukomanTT in #942
• Refactor into call claude by @vmilosevic in #946
• Update requirements to uplift PyTorch by @mmanzoorTT in #944
• Uplift third_party/tt_forge_models to f224af305a10d38acb9fbd72c0c3514b26ec4544 2026-05-09 by @vvukomanTT in #980
• Add ai model bringup workflow by @vmilosevic in #986
• Ai bringup update by @vmilosevic in #987
• Update ai-bringup workflow by @vmilosevic in #988
• Update ai workflow docker image by @vmilosevic in #989
• Update ai workflow by @vmilosevic in #990
• Update ai workflow by @vmilosevic in #992
• Uplift third_party/tt_forge_models to a64a98131c35b010895198f489355d0e6306934f 2026-05-16 by @vvukomanTT in #993
• Run cpu bringup on builder by @vmilosevic in #994
• Add master ai bringup by @vmilosevic in #1002
• Fix push problem in ai workflows by @vmilosevic in #1003
• Uplift third_party/tt_forge_models to 7201811e7020d0e35e908df47a9e57926ba0aa1c 2026-05-23 by @vvukomanTT in #1005

New Contributors

• @zoecarver made their first contribution in #925
• @abrown made their first contribution in #934
• @mmanzoorTT made their first contribution in #944

Full Changelog: 1.0.0...1.2.0

---

LLM Performance

Model	Token/sec/user	Batch	Token/sec	ttft (ms)
facebook/opt-125m	6.0	1	6.0	175.07
pytorch_Falcon_3_1B_Base_nlp_causal_lm_huggingface	57.0	32	1824.0	281.39
pytorch_Falcon_3_3B_Base_nlp_causal_lm_huggingface	37.0	32	1184.0	385.3
pytorch_Gemma_1.1_2B_IT_nlp_causal_lm_huggingface	40.0	32	1280.0	428.1
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface	22.0	32	704.0	655.24
pytorch_Llama_3.2_1B_Instruct_nlp_causal_lm_huggingface	68.0	32	2176.0	248.85
pytorch_Mistral_7B_INSTRUCT_v03_nlp_causal_lm_huggingface	20.0	32	640.0	638.69
pytorch_Mistral_Ministral_8B_Instruct_nlp_causal_lm_huggingface	12.0	32	384.0	304.81
pytorch_Phi-1.5_Phi_1_5_nlp_causal_lm_huggingface	24.0	32	768.0	462.4
pytorch_Phi-1_Phi_1_nlp_causal_lm_huggingface	24.0	32	768.0	457.77
pytorch_Phi-2_Phi_2_nlp_causal_lm_huggingface	11.0	32	352.0	1002.6
pytorch_Qwen 2.5_0.5B_Instruct_nlp_causal_lm_huggingface	81.0	32	2592.0	286.17
pytorch_Qwen 2.5_1.5B_Instruct_nlp_causal_lm_huggingface	39.0	32	1248.0	350.69
pytorch_Qwen 2.5_3B_Instruct_nlp_causal_lm_huggingface	33.0	32	1056.0	531.72
pytorch_Qwen 2.5_7B_Instruct_nlp_causal_lm_huggingface	16.0	32	512.0	759.47
pytorch_Qwen 3_0_6B_nlp_causal_lm_huggingface	36.0	32	1152.0	451.46
pytorch_Qwen 3_1_7B_nlp_causal_lm_huggingface	30.0	32	960.0	490.97
pytorch_Qwen 3_4B_nlp_causal_lm_huggingface	18.0	32	576.0	683.49
pytorch_Qwen 3_8B_nlp_causal_lm_huggingface	13.0	32	416.0	806.92
tiiuae/Falcon3-1B-Base	32.0	1	32.0	50.67

Non-LLM Performance

Model	Batch	Sample/sec
pytorch_BERT_emrecan/bert-base-turkish-cased-mean-nli-stsb-tr_nlp_embed_gen_huggingface	8	44.0
pytorch_BGE-M3_Base_nlp_embed_gen_custom	4	9.0
pytorch_EfficientNet_Timm_B0_cv_image_cls_timm	8	332.0
pytorch_MNIST_Cnn_Dropout_cv_image_cls_custom	32	14688.0
pytorch_MobileNetV2_Mobilenet_v2_cv_image_cls_torch_hub	12	1252.0
pytorch_Qwen 3_Embedding_4B_nlp_embed_gen_huggingface	32	46.0
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface	8	1353.0
pytorch_SegFormer_B0_Finetuned_Ade_512_512_cv_image_seg_huggingface	1	38.0
pytorch_Swin_S_cv_image_cls_torchvision	1	9.0
pytorch_U-Net for Conditional Generation_Base_conditional_generation_huggingface	1	3.0
pytorch_Ultra-Fast Lane Detection v2_TuSimple_ResNet34_Backbone_cv_image_seg_github	1	143.0
pytorch_VGG19-UNet_base_cv_image_seg_custom	1	151.0
pytorch_ViT_Base_cv_image_cls_huggingface	8	237.0
pytorch_VoVNet_Ese_Vovnet19b_Dw.ra_In1k_cv_image_cls_timm	8	713.0

Model coverage

Info: Full list of supported models is available in the assets section.

Model task	Model architecture	Model variant	Model framework	Inference	Training	n150	n300	p150	Single device	Data parallel	Tensor parallel	Model source
conditional generation	U-Net for Conditional Generation	Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	AlexNet	Custom 1x2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
cv image cls	DINOv2	Small	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image cls	EfficientNet	B0	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	MNIST	Cnn Batchnorm	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Dropout	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Dropout	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Nodropout	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom	jax	❌	✅	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom 1x2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
cv image cls	MobileNetV1	Mobilenet v1	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image cls	MobileNetV2	Mobilenet v2	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	ResNet	ResNet50 HuggingFace High Resolution	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	SegFormer	Mit B0	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	Swin	S	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	VGG	HF Vgg19	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	ViT	Base	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image cls	VoVNet	Ese Vovnet19b Dw.ra In1k	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv image seg	MaskFormer Swin-B	Swin Base Coco	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image seg	Ultra-Fast Lane Detection	TuSimple ResNet18 Backbone	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image seg	VGG19-UNet	base	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv img to img	Autoencoder	linear	pytorch	❌	✅	✅	❌	✅	✅	❌	❌	View Source
cv object det	Attention DenseUNet	Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	DETR	ResNet50 Backbone	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	OWL-ViT	Base Patch32	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	PointPillars	pointpillars	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLOP	Default	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLOS Small	Small	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLOv4	Base	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
cv object det	YOLOv7	Default	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv object det	YOLOv9	T	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	ssd512	ssd512	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv panoptic seg	Panoptic Segmentation	ResNet50 Backbone 1x COCO	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm action prediction	OpenVLA-OFT	Finetuned Libero 10	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm action prediction	pi_0	pi0 base	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm image text similarity	CLIP	Base Patch16	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm image text similarity	SigLIP	Base Patch16 224	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
mm visual qa	Mistral	base	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	ALLaM	7B Instruct	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	Command_A_Reasoning	command-a-reasoning-08-2025	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Falcon	3 10B Base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Falcon	3 1B Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Falcon	3 3B Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Falcon	3 7B Base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	GPT-2	Base	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	GPT-2	Xl	jax	❌	✅	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	GPT-OSS	20B	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Gemma	1.1 2B IT	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	1.1 7B IT	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Gemma	2 27B IT	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Gemma	2 2B IT	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	2 9B IT	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Llama	3.1 70B	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Llama	3.1 70B Instruct	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Llama	3.1 8B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Llama	3.2 1B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Llama	3.2 3B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Llama	3.3 70B Instruct	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	7B INSTRUCT v03	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Mistral	Devstral Small 2505	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Large INSTRUCT 2411	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Magistral Small 2506	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Ministral 8B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Mistral	Nemo INSTRUCT 2407	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Mistral	Small 24B INSTRUCT 2501	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Phi-1	Phi 1	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1	Phi 1	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1.5	Phi 1 5	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1.5	Phi 1 5	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-2	Phi 2	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini 128K Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini 4K Instruct	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-4	Phi 4	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Qwen 2	Qwq 32B	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Qwen 2.5	0.5B	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	0.5B Instruct	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	0.5B Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	1.5B Instruct	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source