Light Image Video Generation Inference Framework https://github.com/ModelTC/LightX2V
Fast and Accurate Causal Parallel Decoding using Jacobi Forcing https://arxiv.org/abs/2512.14681
The Anatomy of a Triton Attention Kernel https://arxiv.org/abs/2511.11581
Continuous batching https://huggingface.co/blog/continuous_batching
A lightweight vLLM implementation built from scratch. https://github.com/GeeeekExplorer/nano-vllm
Achieve state of the art inference performance with modern accelerators on Kubernetes https://github.com/llm-d/llm-d
Coral NPU is a hardware accelerator for ML inferencing https://github.com/google-coral/coralnpu
dInfer: An Efficient Inference Framework for Diffusion Language Models https://github.com/inclusionAI/dInfer
Domain specific architectures for AI inference https://fleetwood.dev/posts/domain-specific-architectures
A TensorRT Toolbox for Optimized Large Language Model Inference https://github.com/NVIDIA/TensorRT-LLM
Fast LLM Inference From Scratch https://andrewkchan.dev/posts/yalm.html#section-3.5
There was an error while loading. Please reload this page.