Repository navigation
OjasX: A Math-Level PyTorch Compiler for Low-Memory Training open source CUDA (Looking for Collaborators & TileLang Ideas) #3479
mkrishna793
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi TileLang team and community! 👋
I’m Mohana Krishna, and for the past six months I’ve been independently building OjasX — an open-source, math-level PyTorch training and memory-optimization research backend.
I’m opening this discussion here because TileLang’s approach to explicit tile memory placement and layout scheduling has been a major inspiration for OjasX’s hardware bridge, and I would love to share ideas, get technical feedback from TileLang developers, and explore where math-level autograd rewrites and TileLang kernel lowering can work together.
💡 The Core Idea: Math-Level Reduction Before Kernel Generation
While kernel compilers like TileLang focus on how to tile, schedule, and fuse operations efficiently on hardware, OjasX operates one layer higher — at the reverse-derivative (autograd) and algebraic graph level — to reduce what backpropagation has to store in memory and compute in the first place.
Our goal is to make PyTorch training and fine-tuning practical on memory-constrained consumer GPUs, Apple Silicon unified-memory laptops (
MPS/AMX), and integrated GPUs, where activation tapes, vocabulary logits, and persistent.gradbuffers cause Out-Of-Memory (OOM) crashes long before compute is saturated.Specifically, OjasX combines:
ojasx2/):ReLU,ReLU²,LeakyReLU->32xsmaller than FP32).SiLU,GELU,SwiGLU,GeGLU), trading recomputation for smaller saved activation tapes.rstdretention forRMSNorm, and0fresh saved activation bytes for fixed-tableRoPE(using the inverse rotationholonomic_lm_head_loss): Eliminates materializing full[Batch, Seq, Vocab]logit tensors on the autograd tape.contractions.py,planner.py):T = dY^T @ X) for narrow-rank / bottleneck two-linear chains (linear_chain) without rank approximation..gradHooks (HolonomicGradFreeOptimizer):SGD/AdamW) directly inside PyTorch’sregister_post_accumulate_grad_hookduring backward and immediately freesparam.grad = None(0.00 MBpersistent.gradbuffers).lean/,rocq/, Z3 SMT):csrc/,tile_hal.py) & Experimental TCP Mesh (mesh_hal.py):TORCH_LIBRARY(ojasx, m)C++ operators with AppleAccelerate(cblas_sgemm/vDSP) and Metal (simdgroup_matrix<float, 8, 8>) paths, plus experimental sequential layer partitioning across laptops over TCP (OJXMprotocol with3.76xBlock-INT8 boundary compression).📊 Current Verified Status (
v0.3.0Now Live)We recently completed a full codebase audit and pushed OjasX v0.3.0 with 398 passing
pytestverification tests and reproducible benchmark harnesses (reports/TWO_LAPTOP_PHYSICAL_MESH_REPORT.json&reports/INDUSTRIAL_ALL_LEVEL_BENCHMARK_SCORE.json, tested on Apple M5mps/cpuwith PyTorch 2.8):Dim=1024, Hidden=3072,75.5Mparams):51.1%reduction in fresh autograd-saved storage + persistent.gradbuffers (1,024.3 MB -> 500.6 MB,0.00 MBpersistent.grad, exact loss parity1.054738vs1.054736).[2, 1024, 4096],Vocab=32000):100.0%reduction in fresh saved logit storage on the autograd tape (250.0 MB -> 0.0 MBvia backward chunk replay).[2048, 1024] -> [2048] -> [32]):8.69xspeedup (8.09 ms -> 0.93 ms) and97.7%reduction in fresh saved storage via exact algebraic reassociation.70/70(100%) on sampledF.linearOpInfo tests and25/25(100%) on sampled BackendBenchmm/addmmtrace shapes.(Note on scope: Our Lean/Rocq proofs verify the mathematical specifications, while differential tests validate the Python/C++/Metal runtime. Currently, our compiled native runtime launches CPU and Apple Metal kernels, while CUDA/ROCm/SYCL paths in
tile_hal.pyare source-generation templates.)🔗 Repository: https://github.com/mkrishna793/OjasX-open-source-CUDA
(I am also building NexusSilicon, an open-source EDA tool for chip design.)
🤝 Where OjasX + TileLang Could Collaborate
Right now, OjasX’s
tile_hal.pyemits TileLang-inspired schedules and C++/Metal kernels, but our non-Metal GPU targets (CUDA, ROCm/HIP, SYCL, Vulkan/OpenCL) are currently generated source templates rather than live JIT-launched TileLang kernels.I would love to collaborate with anyone in the TileLang community on:
T = dY^T @ X) directly into TileLangT.Kernel/T.alloc_shared/T.alloc_fragmentschedules.TORCH_LIBRARYdispatcher to TileLang’s native CUDA, ROCm, and CPU codegen backends.ojasx2/tile_hal.py) to general TileLang layout transforms.🙌 Looking for Collaborators & Feedback
While I’ve built the initial compiler, formal proofs, and runtime independently, OjasX needs a community to grow into a mature, cross-vendor ecosystem. If you are interested in:
AOTAutograd,torch.compile, custom C++/Metal/CUDA ops)You don’t need to be a senior compiler expert to contribute — reproducing benchmarks on your laptop, opening issues, improving docs, or sharing mathematical ideas are all hugely welcome.
This project isn’t about attacking any vendor; it’s about giving students, researchers, and indie builders the freedom to train and experiment with AI on the hardware they already own.
I’d love to hear your thoughts, critiques, and ideas! 🚀
All reactions