Sparse Transformers Research

This repository contains JAX research projects for sparse and attention variant transformer models.

• research/st: sparse causal transformer language models.
• research/z1t: Z1-compilable transformers.

Each project has its own pyproject.toml. Install a project from its own directory; see its README for commands.