[Showcase] O(NlogN) Hierarchical VBS-NN: Constant 2.6GB VRAM on ROCm

[ega4l]

I have developed a custom Vertex Byte-Stream Neural Network (VBS-NN) architecture designed for high-efficiency NLP tasks. Unlike traditional transformers that struggle with O(N2) attention complexity, this model utilizes a hierarchical O(NlogN) structure, enabling deep context modeling with a footprint that fits into consumer-grade hardware.

Key Technical Highlights (Running on ROCm RADEON RX 6700 XT):

Constant VRAM: The model maintains a stable 2,601 MB VRAM usage throughout training and generation, regardless of the sequence depth.

Model Weights: ~2.6 GB constant.
Total VRAM Footprint: ~9.7 GB (81% of 12GB).
Context: Includes high-fidelity Logits, Optimizer states, and Gradient buffers for deep hierarchical backpropagation.

Zero Tokenization: Operates directly on raw byte-streams, preserving 100% of data integrity for stylistic and factual accuracy.

AMD Synergy: Leveraging AMD’s compute density via native PyTorch/ROCm.

The "Why":
Coming from a background in optimizing AMD hardware during the mining era, I have a long-standing preference for AMD's compute density and memory architecture. I chose the ROCm stack for this project to demonstrate that a mathematically efficient architecture can achieve state-of-the-art results on consumer-grade AMD GPUs without the need for massive VRAM overhead.

Evidence & Logs:
You can find the detailed training logs and resource utilization plots here:
github - VBS-NN
linedIn

Future Scaling:
I am currently looking to scale this hierarchy to "chapter-length" contexts (100k+ bytes). While the model is stable on my current setup, I am interested in benchmarking this on Radeon™ PRO or Instinct™ hardware to test the upper limits of the O(NlogN) scaling.

I would love to hear feedback from the AMD community or the ROCm engineering team on further optimizing this for the Composable Kernel (CK) or HIP.

[schung-amd]

Do you have any direct comparisons between your work and current state-of-the-art? I'm not entirely convinced by the sample zero-shot generation, I'd like to see how this compares to other methods. Looking forward to future updates.

As for optimization opportunities with the ROCm stack, it's hard to say without seeing the source code, but if you have any specific questions you can submit them as GitHub issues in the appropriate component's repo (or in ROCm/ROCm if you don't know or can't decide) and we'll answer them as best we can.