-
Notifications
You must be signed in to change notification settings - Fork 25.7k
Closed
Labels
module: pipeliningPipeline ParallelismPipeline ParallelismtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
In theory this means that reordering backward calls and then sending their activations could give wrong results. In practice we don't have any tests that seem to do this. But in forward, we store each activation indexed b microbatch_id, and we should probably do the same for backward and add a test for this edge case.
Metadata
Metadata
Assignees
Labels
module: pipeliningPipeline ParallelismPipeline ParallelismtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module