-
Notifications
You must be signed in to change notification settings - Fork 78
Description
Problem description
For many applications, we may want to apply augmentations before (most of the time) or after (seldomly) the adapter has been applied. These can range from noise injections to random selection of subsets of data that should nevertheless keep the batch rectangular. Doing this with the adapter or the simulator placed unnecessary burden and is conceptually misaligned with how practitioners / researchers would go about it. For instance,
- The augmentation may not be necessary during inference
- The augmentation may not be necessary for the validation data
Proposed solution
Each data set object should take optional augmentations that comprise dict[str, Callable] entries, where the dictionary keys match the relevant keys returned by the simulator and the values contain functions applied to the entire batch. Accordingly, workflow objects can accept augmentations as an optional keyword argument that is passed along to the internal data set builders.