Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DART Reduction unreliable with file imported data #1090

Closed
ryanatball opened this issue Nov 22, 2019 · 2 comments
Closed

DART Reduction unreliable with file imported data #1090

ryanatball opened this issue Nov 22, 2019 · 2 comments
Assignees
Labels
Milestone

Comments

@ryanatball
Copy link
Member

@ryanatball ryanatball commented Nov 22, 2019

When importing data into DART from files instead of streaming, decommutation can become many minutes out of order when spread across processes. This isn't inherently wrong, but can cause reduction to think that data is ready to be reduced when it is not all available yet. This change will modify:

  1. Make reduction have a 2 minute holdoff to ensure all data is available
  2. Change decom to a work queue design to ensure that packets do not get multiple minutes out of processing order
  3. Improve decommutation speed using transactions
  4. Don't try to switch to bad configs multiple times
ryanatball added a commit that referenced this issue Nov 22, 2019
@ryanatball

This comment has been minimized.

Copy link
Member Author

@ryanatball ryanatball commented Nov 26, 2019

@aaronatball I believe this fixes your reduction issues

@ryanatball ryanatball closed this Nov 26, 2019
@aaronatball

This comment has been minimized.

Copy link
Contributor

@aaronatball aaronatball commented Nov 26, 2019

@aaronatball I believe this fixes your reduction issues

Awesome, I will check this out the next time I'm doing DART stuff!

@ryanmelt ryanmelt added this to the v4.4.1 milestone Dec 31, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.