1.5.0.dev20260828023103

Installation

Via PyPI

pip install tt-forge==1.5.0.dev20260828023103 --extra-index-url https://pypi.eng.aws.tenstorrent.com/

Via Docker

docker pull ghcr.io/tenstorrent/tt-forge-slim:1.5.0.dev20260828023103

Dependency commits

tt-xla commit: d671e66dd685e26fc4be1a9358cff86ff4629133
tt-mlir commit: e2b21a721955b2b68540d9e6879a215ce3dcfae5
tt-metal commit: d5f9bc4a43b1287523458f1af6a470c6b2adc180

What's Changed

• Uplift third_party/tt_forge_models to 3d755214570c2cca8efd6a7e15ec878f34ad87ba 2026-08-22 by @vvukomanTT in #1026

Full Changelog: 1.5.0.dev20260822000509...1.5.0.dev20260828023103

---

LLM Performance

Model	Token/sec/user	Batch	Token/sec	ttft (ms)	Hardware
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface	9.0	64	576.0	5612.84	n150
pytorch_GPT-OSS_20B_nlp_causal_lm_huggingface	22.0	1	22.0	286.6	p150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface	5.0	32	160.0	10312.07	n150
pytorch_Llama_3.1_70B_Instruct_nlp_causal_lm_huggingface	12.0	32	384.0	1817.29	p150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface	21.0	32	672.0	1244.32	n150
pytorch_Llama_3.1_8B_Instruct_nlp_causal_lm_huggingface	37.0	32	1184.0	567.39	p150

Non-LLM Performance

Model	Batch	Sample/sec	Hardware
pytorch_ResNet_ResNet50_HuggingFace_cv_image_cls_huggingface	8	1352.0	n150

Model coverage

Info: Full list of supported models is available in the assets section.

Model task	Model architecture	Model variant	Model framework	Inference	Training	n150	n300	p150	Single device	Data parallel	Tensor parallel	Model source
nlp causal lm	Qwen 2.5	0.5B Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp embed gen	BERT	emrecan/bert-base-turkish-cased-mean-nli-stsb-tr	pytorch	✅	❌	❌	✅	❌	❌	✅	❌	View Source
mm visual qa	Llama	3.2 11B Vision	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv object det	YOLOP	Default	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm action prediction	OpenVLA-OFT	Finetuned Libero 10	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	SOLAR-10_7B	10 7B Instruct v1.0	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
mm tts	xtts_v2	gpt decode	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm tts	xtts_v2	speaker encoder	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm video ttt	Mochi	Default	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	Source not available
nlp causal lm	GPT-2	Xl	jax	❌	✅	✅	❌	✅	✅	❌	❌	View Source
cv object det	DETR	ResNet50 Backbone	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	2 27B IT	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
cv image cls	AlexNet	Custom 1x2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Qwen 2.5	0.5B Instruct	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
cv image cls	DINOv2	Small	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Nodropout	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 3	0 6B	pytorch	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Falcon	3 1B Base	pytorch	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Llama LoRA	3.2 1B	pytorch	❌	✅	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-3	Mini 128K Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	3B Instruct	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Qwen 2.5 Coder	32B Instruct	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Falcon	3 3B Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	olmo_3	3 32b think	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
cv object det	PointPillars	pointpillars	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	3B Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Falcon	3 10B Base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Mistral	Magistral Small 2506	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	olmo_3	3 7b think	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image seg	Ultra-Fast Lane Detection	TuSimple ResNet18 Backbone	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm image text similarity	CLIP	Base Patch16	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Mistral	Large INSTRUCT 2411	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
cv image seg	MaskFormer Swin-B	Swin Base Coco	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm tts	xtts_v2	gpt latents	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Llama	3.2 1B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Mistral	Ministral 8B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Qwen 2.5 Coder	0.5B	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
cv image seg	VGG19-UNet	base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
mm visual qa	Llama	3.2 11B Vision Instruct	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-4	Phi 4	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp embed gen	Qwen 3	Embedding 4B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Llama	3.1 70B Instruct	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Llama	3.1 8B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
cv image cls	MNIST	Cnn Dropout	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	GPT-OSS	20B	pytorch	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Llama	3.1 70B	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
nlp causal lm	Phi-3	Mini 4K Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-2	Phi 2	jax	✅	❌	❌	✅	❌	❌	❌	✅	View Source
nlp causal lm	Qwen 2.5	1.5B Instruct	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Phi-3	Mini Instruct	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	14B Instruct	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
mm tts	xtts_v2	gpt prefill	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Cnn Batchnorm	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv img to img	Autoencoder	linear	pytorch	❌	✅	✅	❌	✅	✅	❌	❌	View Source
nlp embed gen	Qwen 3	Embedding 8B	pytorch	✅	❌	✅	✅	✅	✅	❌	✅	View Source
cv object det	YOLO-World	Small 640	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5 Coder	1.5B	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	arcee_ai	arcee Spark	pytorch	✅	❌	❌	❌	✅	✅	❌	❌	View Source
cv image cls	ResNet	ResNet50 HuggingFace High Resolution	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	0.5B	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
nlp causal lm	Mistral	Nemo INSTRUCT 2407	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
mm image text similarity	SigLIP	Base Patch16 224	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
conditional generation	U-Net for Conditional Generation	Base	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	Swin	S	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Phi-1	Phi 1	jax	✅	❌	✅	✅	✅	✅	❌	✅	View Source
mm visual qa	Mistral	base	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
cv object det	YOLOS Small	Small	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp token cls	BiLSTM-CRF	Default	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Gemma	1.1 7B IT	pytorch	✅	❌	❌	✅	✅	✅	❌	✅	View Source
nlp causal lm	Llama	3.2 3B	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Qwen 2.5	72B	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
cv image cls	ViT	Base	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
nlp causal lm	Gemma LoRA	1.1 2B IT	pytorch	❌	✅	✅	❌	✅	✅	❌	❌	View Source
cv image cls	MNIST	Mlp Custom	jax	✅	❌	✅	❌	✅	✅	❌	❌	View Source
nlp causal lm	Command_A_Reasoning	command-a-reasoning-08-2025	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
cv object det	YOLOv7	Default	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source
cv image cls	SegFormer	Mit B0	pytorch	✅	❌	✅	✅	✅	✅	✅	❌	View Source
nlp causal lm	Llama	3.3 70B Instruct	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	View Source
mm video ttt	HunyuanVideo15	Transformer	pytorch	✅	❌	❌	❌	❌	❌	❌	✅	Source not available
cv image cls	VGG	HF Vgg19	pytorch	✅	❌	✅	❌	✅	✅	❌	❌	View Source