Skip to content
#

sdpa

Star

Here are 2 public repositories matching this topic...

Language: Python
Filter by language
All 5 Python 2 Cuda 1 HTML 1 MATLAB 1

aidendorian / Marcella-60M-SLM

Star 2

A 66M parameter decoder-only transformer language model implemented from scratch in PyTorch. Features a custom SentencePiece tokenizer, RoPE positional embeddings, SwiGLU feed-forward network, per-layer KV cache for efficient autoregressive inference, and a Svelte-based streaming chat interface.

transformers torch pytorch pretrained-models slm language-model rope alpaca sdpa finetuning sentencepiece transformer-models kv-cache small-language-models fineweb flashattention marcella
  • Updated Apr 5, 2026
  • Python

Improve this page

Add a description, image, and links to the sdpa topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the sdpa topic, visit your repo's landing page and select "manage topics."

Learn more