Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threshold should only produce positive updates #2466

Merged
merged 2 commits into from Mar 28, 2020

Conversation

frankmcsherry
Copy link
Contributor

@frankmcsherry frankmcsherry commented Mar 28, 2020

Threshold would set negative diffs to zero, but that confused other parts of the system. Specifically (for future ref) differential's consolidate method, which we used, doesn't seem to expect zero diffs and does not proactively filter out those that it does not create.

Fixes #2464

@benesch
Copy link
Member

benesch commented Mar 28, 2020

Hang on, this is a differential bug by my reckoning.

@benesch
Copy link
Member

benesch commented Mar 28, 2020

I have an alternative fix coming in shortly and then we can discuss!

@frankmcsherry
Copy link
Contributor Author

Hang on, this is a differential bug by my reckoning.

You'll have to convince me. :) I can see it being a doc bug, but I think consolidate is working as intended (just, it doesn't state its prereqs well enough).

@benesch
Copy link
Member

benesch commented Mar 28, 2020

TimelyDataflow/differential-dataflow#264

The docs on consolidate state:

/// This method will sort `vec[offset..]` and then consolidate runs with the same first
/// element of a pair by accumulating the second elements of the pairs. Should the final
/// accumulation be zero, the element is discarded.

And yet if the slice has one element like [("a", 0)] consolidate will fail to remove that element.

Copy link
Member

@benesch benesch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well do this too!

@frankmcsherry frankmcsherry merged commit 20820c0 into MaterializeInc:master Mar 28, 2020
@cuongdo
Copy link
Contributor

cuongdo commented Mar 31, 2020

@frankmcsherry could you add a relevant release note to doc/user/release-notes.md?

@frankmcsherry
Copy link
Contributor Author

Tbh, I'm not sure what to add. There was a bug that caused a crash, and it doesn't crash any more. There should be zero user-facing changes, other than fewer crashes. Any tips?

@frankmcsherry
Copy link
Contributor Author

Looking at the description of the file, I don't think this qualifies as a "major bug fix". I'm up for being corrected on that, though.

@frankmcsherry frankmcsherry deleted the fix_2464 branch February 11, 2022 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

crash: constant folding encountered reduce on collection with non-positive multiplicities
3 participants