Running LLMs on PyTorch with ROCm

[iswaryaalex]

This tutorial is to quickly bringup user using a super easy example

1. Setup user environment with Rocm and Pytorch on windows/Linux
2. Run LLMs locally. We have tried GPT-OSS-20B and Mistral 7B. Script is straightforward to run
3. Building on that, create a Doc Summarization tool using the LLMs

I have run these examples on my STRX Halo environment.

Need to check if the setup instructions are suitable for Halo Box

[danielholanda]

@bbrosenba , @ChineduN for visibility

[jammm]

I have run these examples on my STRX environment.

A test in strix halo would be good.

[jammm]

Looks good overall. Check my comments re. environment setup for README.md.
Please test on strix halo for both Windows and Linux.

[iswaryaalex]

I have run these examples on my STRX environment.

A test in strix halo would be good.

Done, Strix Halo Windows and Linux Tested.

[iswaryaalex]

@danielholanda The team reviewed and clearly highlighted the installation clarity needed in pytorch installs for ROCm.
I used builds based on this doc here https://rocm.docs.amd.com/en/7.10.0-preview/rocm-for-ai/pytorch-comfyui.html#install-pytorch that highlights python -m pip install --index-url https://repo.amd.com/rocm/whl/gfx1151/ torch torchvision torchaudio for Windows and Linux

[DavidPerryamd]

@danielholanda The team reviewed and clearly highlighted the installation clarity needed in pytorch installs for ROCm. I used builds based on this doc here https://rocm.docs.amd.com/en/7.10.0-preview/rocm-for-ai/pytorch-comfyui.html#install-pytorch that highlights python -m pip install --index-url https://repo.amd.com/rocm/whl/gfx1151/ torch torchvision torchaudio for Windows and Linux

This string is correct here^
but the snippet in platform.md references a pytorch.org repo, is this a mistake?

[iswaryaalex]

@danielholanda The team reviewed and clearly highlighted the installation clarity needed in pytorch installs for ROCm. I used builds based on this doc here https://rocm.docs.amd.com/en/7.10.0-preview/rocm-for-ai/pytorch-comfyui.html#install-pytorch that highlights python -m pip install --index-url https://repo.amd.com/rocm/whl/gfx1151/ torch torchvision torchaudio for Windows and Linux

This string is correct here^ but the snippet in platform.md references a pytorch.org repo, is this a mistake?

@DavidPerryamd, to avoid duplication, we have this for https://github.com/amd/halo_playbooks/blob/e4aa27c81e43a0f411288d28b65a827ab594531b/playbooks/dependencies/pytorch.md pytorch installation

[adamlam2-amd]

Hey - I'm Adam from the solutions team, and I'll be helping with UX Review.

Good job overall - just a couple more minor things that I commented.

The main question I have is: do users have access to the full files in assets/? If not, we need to provide them the full code on the website. This can be provided either by: 1 long code snippet (with copy button), or, preferably, several code snippets that guide the user through each important learning goal.
- ex. run_llm.py is pretty simple and straightforward. Maybe we can shorten the code (lines-wise) and put it all in one big code block.
- summarizer.py is more complex - we could separate it into a couple sections with comments, and explain key parts, like the system prompt,

[iswaryaalex]

The main question I have is: do users have access to the full files in assets/? If not, we need to provide them the full code on the website. This can be provided either by: 1 long code snippet (with copy button), or, preferably, several code snippets that guide the user through each important learning goal. - ex. run_llm.py is pretty simple and straightforward. Maybe we can shorten the code (lines-wise) and put it all in one big code block. - summarizer.py is more complex - we could separate it into a couple sections with comments, and explain key parts, like the system prompt,

@adamlam2-amd To give valuable learning and one click try to the user, would be great to retain the code snippet as short as it already is
Do you think we can embed the code snippet like this in my screenrecording? To allow user to copy/download

Recording.2026-01-23.154156.mp4

[danielholanda]

@jammm We are likely merging very soon. Please let us know if you have any additional recommendations or if this looks good.

[danielholanda]

@adamlam2-amd @iswaryaalex Added the code preview feature directly to this PR

[jammm]

LGTM after the feedback is addressed. The part about screenshots would be nice to have but not critical for initial release so is optional.

[adamlam2-amd]

Thanks for all the hard work @iswaryaalex . Thanks @danielholanda for the code preview addition.

[iswaryaalex]

All feedback comments are addressed. Merging this playbook