Whisper Speech Recognition on AMD ROCm (gfx1151/Strix Halo)

Ultra-fast speech recognition using OpenAI's Whisper model with CTranslate2 GPU acceleration on AMD Radeon 8060S Graphics (gfx1151).

🏆 Performance Highlights

Achieved on AMD Radeon 8060S (gfx1151) - Strix Halo:

CTranslate2 Tiny Model + GPU: 138x faster than real-time (11s audio in 80ms)
CTranslate2 Large-v3 Model + GPU: 9x faster than real-time (11s audio in 1.22s) 🏆
whisper.cpp Large-v3 Model + GPU: 4.45x faster than real-time (11s audio in 1.27s)
CTranslate2 Large-v3 Model + CPU: 4.06x faster than real-time (11s audio in 2.71s)

See RESULTS.md for detailed benchmarks.

Features

✅ GPU Acceleration on AMD Radeon 8060S (gfx1151) via ROCm 6.4.3
⚡ 138x real-time performance with CTranslate2 tiny model
🏆 9x real-time with large-v3 (2x faster than whisper.cpp on same hardware)
🎯 Perfect accuracy on test audio (JFK speech)
🔧 From-scratch build instructions for gfx1151 support
📊 Multiple models (tiny to large-v3) and quantization options
🐳 Docker support (ROCm 6.1, limited to CPU for gfx1151)
💻 Local build (ROCm 6.4.3+, full GPU support)

Quick Start

Prerequisites

Before starting, ensure you have:

ROCm 6.4.3+ installed
uv installed: curl -LsSf https://astral.sh/uv/install.sh | sh
Build tools: sudo pacman -S base-devel cmake git (Arch) or sudo apt install build-essential cmake git (Ubuntu)

Automated Setup (Recommended)

Run the bundled helper to build HIP-enabled CTranslate2, install the Python wheel, convert whisper-large-v3, and execute the GPU smoke test:

UV_CACHE_DIR=$PWD/.cache/uv ./scripts/uv_ct2_gpu_test.sh

The script keeps all build artefacts under .deps/ and reuses the existing .venv; see scripts/uv_ct2_gpu_test.sh for environment overrides.

Installation Steps

# 1. Clone this repository
git clone https://github.com/yourusername/whisper-rocm.git
cd whisper-rocm

# 2. Install system dependencies
# For Arch Linux:
sudo pacman -S rocm-hip-sdk rocrand rocthrust hipcub hiprand miopen-hip composable-kernel hipblaslt

# For Ubuntu/Debian:
sudo apt install rocm-dev rocm-libs rocrand rocthrust hipcub hiprand miopen-hip

# 3. Build CTranslate2 from source
# Clone CTranslate2
git clone --recursive https://github.com/OpenNMT/CTranslate2.git
cd CTranslate2
git checkout v3.23.0
git submodule update --init --recursive

# Build and install (this creates ~/ctranslate2-install)
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release \
      -DWITH_CUDA=ON \
      -DWITH_CUDNN=ON \
      -DGPU_RUNTIME=HIP \
      -DCMAKE_HIP_ARCHITECTURES=gfx1151 \
      -DGPU_TARGETS=gfx1151 \
      -DCMAKE_INSTALL_PREFIX=$HOME/ctranslate2-install \
      -DOPENMP_RUNTIME=COMP \
      -DWITH_MKL=OFF \
      -DWITH_DNNL=OFF \
      -DBUILD_CLI=OFF \
      ..
make -j$(nproc)
make install

# Return to whisper-rocm directory
cd ../..  # back to whisper-rocm

# 4. Create Python environment with uv
uv venv --python 3.12 .venv
source .venv/bin/activate

# 5. Install Python dependencies
uv pip install torch --index-url https://download.pytorch.org/whl/rocm6.2
uv pip install transformers soundfile

# 6. Install CTranslate2 Python bindings
cd CTranslate2/python
export CT2_INSTALL_PREFIX=$HOME/ctranslate2-install
export CPLUS_INCLUDE_PATH=$CT2_INSTALL_PREFIX/include
export LIBRARY_PATH=$CT2_INSTALL_PREFIX/lib
uv pip install --no-build-isolation .
cd ../..  # back to whisper-rocm

# 7. You're ready! Use the helper script
./run_whisper.sh jfk.wav tiny

Installation time: ~15-20 minutes (including downloads)

See INSTALL.md for complete step-by-step instructions.

Requirements

Hardware

GPU: AMD Radeon 8060S (gfx1151) or compatible AMD GPU
RAM: 16GB+ recommended
Storage: ~10GB for dependencies and models

Software

OS: Linux (tested on Arch Linux)
ROCm: 6.4.3 or newer (required for gfx1151 support)
Python: 3.10-3.13 (3.14 not yet supported by PyTorch)

Critical Dependencies

miopen-hip - ROCm's cuDNN equivalent (~8GB installed)
rocrand, rocthrust, hipcub, hiprand - ROCm compute libraries
ROCm 6.4.3+ with gfx1151 TensileLibrary support

Usage

Activate Environment

# Activate Python virtual environment
source .venv/bin/activate

# CRITICAL: Force system ROCm libraries (required to avoid PyTorch library conflicts)
export ROCBLAS_TENSILE_LIBPATH=/opt/rocm/lib/rocblas/library
export LD_LIBRARY_PATH=/opt/rocm/lib:$HOME/ctranslate2-install/lib:$LD_LIBRARY_PATH

⚠️ Important: The environment variables are critical for GPU acceleration. PyTorch bundles its own ROCm libraries that don't support gfx1151. Setting these variables forces the use of system ROCm libraries that do support gfx1151.

Convert a Whisper Model

# Convert tiny model (fastest, 75MB)
ct2-transformers-converter --model openai/whisper-tiny \
    --output_dir whisper-tiny-ct2 --quantization int8

# Convert large-v3 model (best quality, 3GB)
ct2-transformers-converter --model openai/whisper-large-v3 \
    --output_dir whisper-large-v3-ct2 --quantization float16

Transcribe Audio

Easy way (recommended - uses helper script with correct environment):

# Make script executable (first time only)
chmod +x run_whisper.sh

# Test with tiny model (fastest)
./run_whisper.sh jfk.wav tiny

# Test with large-v3 model (best quality)
./run_whisper.sh jfk.wav large-v3

# Test with your own audio file
./run_whisper.sh your_audio.mp3 large-v3

Manual way (if you need more control):

# Set environment variables first
export ROCBLAS_TENSILE_LIBPATH=/opt/rocm/lib/rocblas/library
export LD_LIBRARY_PATH=/opt/rocm/lib:$HOME/ctranslate2-install/lib:$LD_LIBRARY_PATH
source .venv/bin/activate

# Then run the test script
python test_ct2_full.py jfk.wav tiny
python test_ct2_full.py jfk.wav large-v3

Example Output

============================================================
CTranslate2 Whisper Test - AMD ROCm GPU
============================================================

CTranslate2 version: 3.23.0
CUDA available: True
CUDA device count: 1

============================================================
Transcribing: jfk.wav
Device: cuda
============================================================

Loading audio file...
Audio loaded in 0 ms (duration: 11.00s, sample rate: 16000)
Loading WhisperProcessor...
Computing audio features...
Features computed in 19 ms
Loading model from whisper-tiny-ct2...
Model loaded in 326 ms on cuda (compute_type=int8_float16)
Detecting language...
Detected language: en (97.95%) in 332 ms
Transcribing...

============================================================
RESULTS
============================================================
Transcription: And so my fellow Americans ask not what your
country can do for you, ask what you can do for your country.

Performance Metrics:
  Audio duration: 11.00s
  Transcription time: 84 ms
  Real-time factor: 0.008x
  Speed: 130.41x faster than real-time

Breakdown:
  Audio load:           0 ms
  Features:            19 ms
  Model load:         326 ms
  Language detect:    332 ms
  Transcribe:          84 ms
  Total:              760 ms (0.76s)
============================================================

Model Options

Model	Size	GPU VRAM	Speed (gfx1151)	Accuracy	Use Case
tiny	75 MB	~500 MB	138x real-time	Good	Real-time apps
base	150 MB	~700 MB	~80x real-time	Better	Fast transcription
small	500 MB	~1.5 GB	~40x real-time	Great	Balanced
medium	1.5 GB	~3 GB	~15x real-time	Excellent	High quality
large-v3	3 GB	~6 GB	9x real-time	Best	Production

Recommendation for gfx1151:

Real-time apps: tiny or base model (138x - 80x real-time)
Batch processing: small or medium model (40x - 15x real-time)
High accuracy: large-v3 model (9x real-time, best quality)

Quantization Options

int8: Fastest, smallest memory, slight quality reduction
int8_float16: Best for GPU (int8 weights, float16 compute)
float16: Excellent quality, 2x faster than float32
float32: Highest quality, slowest

Architecture Support

✅ Supported: Local Build (ROCm 6.4.3+)

gfx1151 (Radeon 8060S - Strix Halo) - This guide
gfx1100-1102 (RDNA 3)
gfx900-942 (MI series, Vega, RDNA 1-2)

❌ Not Supported: Docker (ROCm 6.1)

gfx1151 missing from rocBLAS TensileLibrary
Falls back to CPU-only mode

Performance Comparison

Full comparison with whisper.cpp: RESULTS.md

Implementation	Device	Model	Total Time	Speed
CTranslate2	GPU	Tiny	0.41s	138x 🏆
CTranslate2	GPU	Large-v3	4.09s	9x 🥇
whisper.cpp	GPU	Large-v3	2.47s	4.45x
CTranslate2	CPU	Large-v3	7.31s	4.06x

Project Structure

whisper-rocm/
├── README.md                    # This file
├── INSTALL.md                   # Detailed installation guide
├── RESULTS.md                   # Performance benchmarks
├── quick_install.sh             # Automated installation script
├── run_whisper.sh              # Helper script with correct environment (recommended)
├── test_ct2_full.py            # GPU transcription test
├── test_large_v3.py            # Large-v3 specific test
├── test_ct2_gpu.py             # GPU detection test
├── requirements.txt            # Python dependencies
├── jfk.wav                      # Test audio file
├── .venv-convert/              # Python 3.12 environment (for model conversion)
├── docker_rocm/
│   ├── Dockerfile.rocm         # Docker build (ROCm 6.1, CPU only for gfx1151)
│   └── ct2_3.23.0_rocm.patch   # ROCm compatibility patch
└── ctranslate2-install/        # Installed library (after build)
    ├── lib/
    ├── include/
    └── ...

Troubleshooting

GPU Not Detected

# Check ROCm
rocm-smi

# Check gfx1151 support
ls /opt/rocm/lib/rocblas/library/ | grep gfx1151

# Test GPU detection
python -c "import ctranslate2; print(ctranslate2.get_cuda_device_count())"

PyTorch ROCm Library Conflict (CRITICAL)

If you see errors like Cannot read TensileLibrary.dat for GPU arch : gfx1151 or the GPU is not detected:

Problem: PyTorch bundles its own ROCm libraries that don't support gfx1151. When transformers loads PyTorch, it uses these incompatible libraries instead of your system ROCm.

Solution: Force system ROCm libraries to be loaded first:

# Set BEFORE running any Python scripts
export ROCBLAS_TENSILE_LIBPATH=/opt/rocm/lib/rocblas/library
export LD_LIBRARY_PATH=/opt/rocm/lib:$HOME/ctranslate2-install/lib:$LD_LIBRARY_PATH

# Then activate your environment
source .venv/bin/activate

# Now run your script
python test_ct2_full.py jfk.wav large-v3

Permanent fix (add to ~/.bashrc or ~/.zshrc):

export ROCBLAS_TENSILE_LIBPATH=/opt/rocm/lib/rocblas/library
export LD_LIBRARY_PATH=/opt/rocm/lib:$HOME/ctranslate2-install/lib:$LD_LIBRARY_PATH

Conv1D GPU Error

If you see: Conv1D on GPU currently requires the cuDNN library

Solution: Install MIOpen

# Arch Linux
sudo pacman -S miopen-hip composable-kernel hipblaslt

# Ubuntu
sudo apt install miopen-hip

Then rebuild CTranslate2 with -DWITH_CUDNN=ON.

See INSTALL.md for more troubleshooting.

Build Requirements

To build CTranslate2 with gfx1151 GPU support:

ROCm 6.4.3+ (gfx1151 support in rocBLAS)
System packages: miopen-hip, rocrand, rocthrust, hipcub, hiprand
CMake flags:

cmake -DCMAKE_BUILD_TYPE=Release \
      -DWITH_CUDA=ON \
      -DWITH_CUDNN=ON \
      -DGPU_RUNTIME=HIP \
      -DCMAKE_HIP_ARCHITECTURES=gfx1151 \
      -DGPU_TARGETS=gfx1151 \
      -DOPENMP_RUNTIME=COMP \
      ..

See INSTALL.md for complete build instructions.

Documentation

INSTALL.md - Complete installation guide from scratch
RESULTS.md - Performance benchmarks and comparisons
quick_install.sh - Automated installation script
AGENTS.md - Contributor workflow and coding guidelines

Why This Guide?

This guide was created to enable GPU acceleration on the AMD Radeon 8060S (gfx1151/Strix Halo), which is:

❌ Not supported by Docker ROCm 6.1 images
✅ Fully supported by local ROCm 6.4.3 installation

The key insight: whisper.cpp worked out-of-the-box because it was compiled against local ROCm 6.4.3, which has gfx1151 support. CTranslate2 needed to be built from source with the same local ROCm to achieve GPU acceleration.

Related Projects

CTranslate2 - Fast inference engine
whisper.cpp - Alternative implementation
OpenAI Whisper - Original model
ROCm CTranslate2 - AMD's fork

References

License

This project follows the same license as CTranslate2 (MIT License).

Acknowledgments

AMD ROCm team for the CTranslate2 blog post and Docker images
OpenNMT team for CTranslate2
OpenAI for the Whisper model

Last Updated: 2025-10-09 Tested on: Arch Linux, ROCm 6.4.3, AMD Radeon 8060S (gfx1151) Status: ✅ Working with GPU acceleration

Name		Name	Last commit message	Last commit date
docker_rocm		docker_rocm
scripts		scripts
.gitignore		.gitignore
AGENTS.md		AGENTS.md
INSTALL.md		INSTALL.md
README.md		README.md
RESULTS.md		RESULTS.md
amd.txt		amd.txt
jfk.wav		jfk.wav
quick_install.sh		quick_install.sh
requirements.txt		requirements.txt
run_rocm_test.sh		run_rocm_test.sh
run_whisper.sh		run_whisper.sh
test_ct2_full.py		test_ct2_full.py
test_ct2_gpu.py		test_ct2_gpu.py
test_faster_whisper.py		test_faster_whisper.py
test_large_v3.py		test_large_v3.py
test_whisper_rocm.py		test_whisper_rocm.py

Latest commit

History

Folders and files

Repository files navigation