A minimal GPU implementation in SystemVerilog, built entirely from scratch to explore how GPUs work at a hardware level.
This project expands upon the original tiny-gpu idea with new architectural modules, an improved verification environment, and a proper C++/Verilator-based simulation flow.
- gpu-extended-from-tiny
Modern GPUs are built around massive parallelism, but the hardware-level design principles behind them are rarely documented publicly. While GPU programming guides are common, open hardware explanations are not.
This project aims to fill that gap — it’s a fully working, modular GPU written in SystemVerilog, demonstrating how SIMD architectures execute kernels, manage memory, and handle branching.
It extends the original tiny-gpu in several major ways:
- ✅ New architectural modules: scheduler redesign, branch prediction, enhanced memory controller
- ✅ Modular hierarchy with
dut/srcstructure and unified Makefile build system - ✅ C++ simulation harness with multiple test configurations
- ✅ Integrated UVM-style verification using cocotb and pyuvm
- ✅ Configurable logging and debugging through
DEBUG_LOGand runtime arguments
The GPU executes one kernel at a time.
Before execution begins:
- The kernel is loaded into program memory
- Input data is loaded into data memory
- The control register sets the total number of threads and start signal
Main components:
- Device Control Register (DCR)
- Dispatcher
- Compute Cores
- Memory Controllers
- (WIP) Cache layer
- (Planned) Rasterization Unit — to convert pixel fragments into screen-space output in future versions
Stores kernel configuration parameters such as the total thread count, block size, and the kernel launch signal.
This module acts as the GPU’s “control interface”.
Distributes thread blocks across available cores and monitors execution progress.
It supports multi-core synchronization and balanced block assignment to ensure optimal resource use.
- Data Memory: 8-bit address × 8-bit data (256 entries)
- Program Memory: 8-bit address × 16-bit instruction width (256 × 16)
Coordinate all requests between compute cores and memory.
The design supports concurrent instruction and data access, request queuing, and bandwidth throttling.
Recent work: Added independent control logic for instruction and data channels, preparing the design for cache integration.
A future feature that will cache frequently accessed data to reduce latency and global memory pressure.
Each core executes one block of threads at a time.
Each thread has its own ALU, LSU, Program Counter, and Register File.
Manages instruction issue across all threads in a block.
I’ve extended this with a state-based scheduler to support warp-level execution and branch divergence handling later on.
Fetches the current instruction from program memory (and soon, from the instruction cache).
Decodes the instruction into control signals for execution units.
Each thread has a dedicated register file, including:
- General-purpose registers (
R0–R12) - Read-only special registers (
%blockIdx,%blockDim,%threadIdx)
Perform integer arithmetic and comparison operations:
ADD,SUB,MUL,DIV,CMP
The ALU now supports signed arithmetic and has been prepared for pipelining.
Handle memory load/store (LDR, STR) operations asynchronously through the memory controller.
Added stall detection and wait-state signaling.
Program counters now hold both the current instruction address and NZP flags for branching (BRnzp).
Recently extended with a basic branch prediction mechanism that reduces unnecessary stalls and logs prediction outcomes.
Implements an 11-instruction custom ISA supporting both arithmetic and control flow:
BRnzp— Conditional branchCMP— Comparison instruction (sets NZP flag)ADD,SUB,MUL,DIV— ArithmeticLDR,STR— Memory operationsCONST— Load constantRET— Return from kernel
Registers R0–R12 are general-purpose, while the last three are dedicated to thread/block indices.
- FETCH — Load instruction from program memory
- DECODE — Generate control signals
- REQUEST — Send memory request (if any)
- WAIT — Handle stalls or latency
- EXECUTE — Perform ALU operation
- UPDATE — Write back to registers
Each thread executes this flow on its own data using the SIMD (Single Instruction, Multiple Data) model.
All threads within a block run in sync, sharing control but processing different data.
Adds two 1×8 matrices in parallel.
Each thread performs one element-wise addition.
.threads 8
.data 0 1 2 3 4 5 6 7
.data 0 1 2 3 4 5 6 7
...
RETMultiplies two 2×2 matrices using nested loops and branching.
.threads 4
.data 1 2 3 4
.data 1 2 3 4
...
RETSimulation is handled through a unified Makefile flow using Verilator and a C++ harness.
- Per-core and per-thread logging (
DEBUG_LOG) - Multiple test configurations (
+test=<name>) - Parallel multi-core dispatch
- Adjustable runtime parameters
# Build & run a test
make run DEBUG=1 +test=matadd
# Run a specific branch prediction test
make run DEBUG=1 +test=branch_pred_even_odd
# Disable logging for speed
make run DEBUG=0
Verification uses a UVM-style environment built on cocotb + pyuvm.
- Monitors track instruction fetch/decode/writeback
- Scoreboard validates final register and memory states
- Coverage bins record instruction types, branch outcomes, and memory access patterns
- Parallel tests run automatically via the Makefile
This setup ensures correctness during ongoing development of the scheduler, memory controller, and branch predictor.
Future versions will include a multi level cache hierarchy and shared memory for intra block communication.
Merging consecutive memory accesses across threads to improve bandwidth utilization.
Allowing overlapping instruction execution stages for higher throughput.
Executing multiple thread groups (warps) in parallel to hide latency.
Handling per thread control flow differences with reconvergence logic.
Allowing threads to synchronize at specific points before continuing execution.
Planned updates and future work:
- Integrate instruction cache
- Extend branch prediction with accuracy tracking
- Finalize LSU stall management
- Add lightweight core pipelining
- Implement memory coalescing
- Add optional floating-point support
- Optimize Verilator trace performance
- Expand cocotb test coverage
- Add a rasterization unit for basic graphics functionality and pixel-level computation
gpu-extended-from-tiny is an evolving educational GPU design that bridges architecture and verification.
It combines real hardware concepts thread scheduling, branch prediction, and memory management with a modern verification flow, providing both a learning tool and a foundation for future GPU extensions like rasterization and graphics acceleration.