Skip to content

Mix real code-switching corpora into training, not just evaluation #7

Description

@Awshesh12

All our training data is synthetic. Real code-switched speech is used only for evaluation. The paper's central finding is that this costs a lot of generalisation — the obvious next move is to close the gap from the training side.

Candidate sources

  • CS-FLEURS xtts/train — 2097 JA–EN rows already available via data/load_csfleurs.py, currently unused. Note these are TTS, so they help with switch-point modelling but not acoustic realism.
  • CS-FLEURS read — real voices, but it is our test set. Would require carving out a disjoint train split and re-freezing the benchmark, which invalidates comparison with the current paper. Do not do this casually.
  • SEAME — Mandarin–English, wrong pair, but usable for a cross-lingual transfer experiment
  • Japanese monolingual real speech (JSUT, Common Voice ja) mixed with English (LibriSpeech, Common Voice en) — cheap, real acoustics, and the composition ablation in Table 6 already shows monolingual data punches above its weight

Experiment to run

A curve: hold total training volume fixed and vary the synthetic:real ratio (100:0, 75:25, 50:50, 25:75). How much real data is needed before real-speech ScriptAcc saturates? That result would be a genuine contribution on its own and directly extends §8.

Watch out

Any use of CS-FLEURS in training breaks the clean held-out property the current benchmark depends on. Document clearly whichever way it goes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    dataCorpora, augmentation, dataset constructiontrainingFine-tuning recipes, ablations, hyperparameters

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions