Skip to content

Releases

KuralHub v1.0.0: Multilingual Speech Emotion Recognition Datasets and Benchmark (Interspeech 2026)

Choose a tag to compare

@Luxshan2000 Luxshan2000 released this 31 Oct 07:22
fbaedd9

First public release of KuralHub, the code, data and website for the Interspeech 2026 paper KuralHub: Exposing Typological Capability Frontiers in Multilingual Speech Emotion Recognition by Luxshan Thavarasa*, Jubeerathan Thevakumar*, Thanikan Sivatheepan and Uthayasanker Thayasivam (University of Moratuwa, Sri Lanka). *Equal contribution.

KuralHub is an open catalog of speech emotion recognition (SER) datasets for the 70 most widely spoken languages, and a benchmark of 11 self-supervised speech models (HuBERT, wav2vec 2.0, WavLM and Whisper) on 33 datasets in 29 languages.

Links

What is in this release

  • Dataset survey. 90 speech emotion datasets across the 70 most widely spoken languages. 45 languages have at least one dataset and 25 have none we could find. Each dataset in datasets/<Language>/ lists its emotion labels, size, speakers, recording setup, licence, access link and BibTeX.
  • Benchmark. 363 training runs: 11 frozen pretrained encoders with a two-layer classification head on 33 datasets. Every run has its classification reports, confusion matrices and loss curve in results/train_val_test_logs/.
  • Code. Dataset download, metadata extraction and training in src/. Requires Python 3.12 or newer.
  • Data files, attached below and also on the website:
    • runs.csv and runs.json: one row per run with validation and test accuracy, macro and weighted F1, macro recall (UAR) and per-emotion F1
    • datasets.json: every catalogued dataset with language, emotion labels, size, access, licence and links
    • languages.json: all 70 languages with family, ISO code and dataset count

Key findings

  • No model is best for every language. Whisper had the highest mean F1 (0.70), then HuBERT-base (0.68) and WavLM (0.63 to 0.64), but HuBERT won for nine languages and WavLM for English and Swahili.
  • Every model struggles with Dravidian languages. Kannada averaged 0.275 accuracy and Tamil 0.306.
  • More pre-training data is not enough. Mandarin and Indonesian were easy, while French, Spanish and Russian were not.
  • Bigger models did not do better. log(parameters) is negatively correlated with F1 (r = −0.18, p = 0.0007).
  • Fear is the hardest emotion to recognise (F1 0.38). Anger and neutral are the easiest.

Quick start

git clone https://github.com/aaivu/KuralHub.git && cd KuralHub
make setup
make download_dataset    # needs ~/.kaggle/kaggle.json
make meta_extract
DATASET=EMOTA BASE_MODEL=WAVLM_LARGE make train_model

Contribute

25 languages still have no speech emotion dataset we could find, and the benchmark covers 11 models so far. New datasets and models are welcome:

Citation

If you use KuralHub, the benchmark results or the dataset catalog, please cite:

Thavarasa, L., Thevakumar, J., Sivatheepan, T., Thayasivam, U. (2026) KuralHub: Exposing Typological Capability Frontiers in Multilingual Speech Emotion Recognition. Proc. Interspeech 2026, 4673-4678, doi: 10.21437/Interspeech.2026-3502

Official BibTeX from the ISCA Archive, also in CITATION.bib:

@inproceedings{thavarasa26_interspeech,
  title     = {{KuralHub: Exposing Typological Capability Frontiers in Multilingual Speech Emotion Recognition}},
  author    = {Luxshan Thavarasa and Jubeerathan Thevakumar and Thanikan Sivatheepan and Uthayasanker Thayasivam},
  year      = {2026},
  booktitle = {{Interspeech 2026}},
  pages     = {4673--4678},
  doi       = {10.21437/Interspeech.2026-3502},
  issn      = {2958-1796},
}

If you use one of the catalogued datasets, please also cite its original authors. Each dataset file includes their BibTeX.