Perplexity AI open source garden for inference technology
RDMA TransferEngine, P2P MoE dispatch/combine kernel
- Docs: docs/fabric-lib.md
- MLSys'26 paper: fabric-lib: RDMA Point-to-Point Communication for LLM Systems
- Blog Post: RDMA Point-to-Point Communication for LLM Systems
- Blog Post: Enabling Trillion-Parameter Models on AWS EFA
- Blog Post: Weight Transfer for RL Post-Training in under 2 seconds
- Blog Post: Disaggregated Prefill and Decode
Unigram tokenizer encoder
- Docs: docs/unigram.md
- Blog Post: Improving Unigram Tokenizer CPU Performance
Rust and Metal inference server for Qwen3.6-35B-A3B on Apple Silicon. Lily provides greedy text generation through a minimal OpenAI-compatible HTTP API.
- Docs: lily/README.md
- Blog Post: Optimizing On-Device Inference for Apple Silicon
- License: Apache-2.0, with third-party notices in lily/NOTICE
fabric-lib/: RDMA TransferEngine librarylily/: Metal LLM inference for Apple Siliconp2p-all-to-all/: P2P MoE All-to-All implementationpplx-unigram/: Unigram tokenizer encoderpython-ext/: Python extension module from Rust codepython/pplx_garden/: Python code for thepplx_gardenpackagerust/: Rust utility libraries