Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Restrain progress messages #228
This PR ports the logic from #205 to the new event-driven scheduler logic. Most of the logic is unchanged or simplified, due to the state maintained by the new reachability tracker.
One important change is that the default behavior is now with the progress accumulation optimization enabled, rather than disabled as it is in #205. To return to the old behavior, where progress messages are eagerly exchanged rather than exchanged only when justified, set the environment variable
This should cause all progress messages to be exchanged as soon as they are available, even if they cannot obviously improve the progress of the computation.
The down side for non-eager progress message exchange is the potential increase of the critical path for latency sensitive, progress bound computations. Where one
It remains an open question how painful this is, but the alternative (eager progress messages) has a known downside of many orders of magnitude increase in the volume of progress traffic. Ideally, some experimentation will begin to surface any problems, and we can look at ways to fix things further.
Good questions both.
Not planning on landing this without further testing, and perhaps without swapping the defaults back to eager to avoid surprises for folks not paying attention to our public test schedule.