FIGMA: Towards FIne-Grained Music retrievAl (ACL 2026)

Accepted to ACL 2026.

FIGMA retrieves music from natural-language descriptions that specify fine-grained musical attributes — tempo, key, chord progression, beat/time signature — in addition to high-level attributes like genre and mood. This repository contains the inference code for the trained FIGMA model.

• 📄 Paper: https://arxiv.org/abs/2606.06615
• 🌐 Project page: https://nishitanand.github.io/figma-website
• 🤗 Model: https://huggingface.co/nishitanand/FIGMA
• 🎵 Benchmark: https://huggingface.co/datasets/nishitanand/FGMCaps-benchmark
• 📚 Data: https://huggingface.co/datasets/nishitanand/FGMCaps-data

Installation

pip install -r requirements.txt
# MuQ is installed via:
pip3 install muq

The first run downloads the MuQ and E5 encoder definitions from HuggingFace. The FIGMA checkpoint bundles the trained weights.

Get the checkpoint

Download the packaged checkpoint from the model repo:

from huggingface_hub import hf_hub_download
ckpt = hf_hub_download(repo_id="nishitanand/FIGMA", filename="figma.ckpt")

Inference

Command line — text → audio retrieval over a folder of clips:

python inference.py --checkpoint figma.ckpt \
    --audio_dir ./clips \
    --query "a song in F minor at 120 BPM in 4/4 time" \
    --topk 5

Command line — dump audio embeddings:

python inference.py --checkpoint figma.ckpt --audio_dir ./clips --save_embeddings emb.pt

Programmatic:

import torch, librosa
from figma_model import Figma, get_tokenizer

device = "cuda"
model = Figma.from_checkpoint("figma.ckpt", device=device)
tok = get_tokenizer()

# text embedding
t = tok(["a jazzy track in C major at 90 bpm"], return_tensors="pt",
        padding="max_length", truncation=True, max_length=128).to(device)
text_emb = model.encode_text(t)                       # [1, 512]

# audio embedding (24 kHz mono -> [B, 1, samples])
wav, _ = librosa.load("clip.wav", sr=24000)
wav = torch.tensor(wav)[None, None].to(device)
audio_emb = model.encode_audio(wav)                   # [1, 512]

score = (audio_emb @ text_emb.T).item()               # cosine similarity

Citation

@inproceedings{figma2026,
  title     = {FIGMA: Towards FIne-Grained Music retrievAl},
  author    = {Anand, Nishit and Seth, Ashish and Ghosh, Sreyan and Manocha, Dinesh and Duraiswami, Ramani},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics},
  year      = {2026},
  url       = {https://arxiv.org/abs/2606.06615}
}

License

The code in this repository is released under the MIT License (see LICENSE). The model checkpoint is released for non-commercial research use under CC BY-NC 4.0, because it bundles the MuQ encoder (weights CC BY-NC 4.0); E5 is MIT. Please cite the paper and attribute MuQ and E5.