data.rst

fairseq.data

Data Loading and Utilities

Datasets

Datasets define the data format and provide helpers for creating mini-batches.

fairseq.data.FairseqDataset

fairseq.data.LanguagePairDataset

fairseq.data.MonolingualDataset

Helper Datasets

These datasets wrap other fairseq.data.FairseqDataset instances and provide additional functionality:

fairseq.data.BacktranslationDataset

fairseq.data.ConcatDataset

fairseq.data.RoundRobinZipDatasets

fairseq.data.TransformEosDataset

Dictionary

fairseq.data.Dictionary

Iterators

fairseq.data.CountingIterator

fairseq.data.EpochBatchIterator

fairseq.data.GroupedIterator

fairseq.data.ShardedIterator