Skip to content

v0.2.1

Choose a tag to compare

@github-actions github-actions released this 27 Oct 17:26
· 2 commits to main since this release
27b35f6

ReactantNitro v0.2.1

Diff since v0.2.0

MLUtils.DataLoader with num_workers now uses its worker processes

A split that is a DataLoader with num_workers > 0 now runs one producer over the loader's own iterate, so MLUtils' worker processes build the batches. Before, the framework fanned out over getobs in its own process and the workers never started.

MLUtils delivers worker batches as they finish, so the order of an epoch does not reproduce. Setup therefore refuses such a split unless it states that:

train = PrefetchIterator(DataLoader(data; batchsize, num_workers = 4); ordered = false)

The binding report shows the split as prefetch: 1 producer, ...; the source parallelizes itself. A source type of your own can opt into the same path by defining ReactantNitro.delegates_parallelism(source) = true.

When using Revise.jl, treat num_workers with care The training process stays on the code a run started with, but worker processes do not: Distributed calls each batch request with invokelatest, so they run the newest definition of a function. If Revise is loaded, saving an edit to getobs, an augmentation or a collate function changes the batches of a run that is already training, from the next batch on, while forward and loss stay on the old code. Do not edit the data pipeline during a run that uses num_workers. The issue is tracked here: MLUtils.jl#267.

Fixes

  • An integer parallel on a DataLoader (a worker-task count) raised a TypeError at setup on the single-producer path.

Compat

  • The MLUtils floor is now 0.4.12.
  • Every stdlib dependency declares compat "1".

Docs

  • A new Data loading page covers DataLoader setup, when to use num_workers, sharing data with worker processes, how a split is driven, and writing a source of your own.

Closed issues:

  • External Dataloader (#56)