Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Guarantee event processing order #1352

Merged
merged 4 commits into from
Apr 14, 2023

Conversation

AndrewSisley
Copy link
Contributor

@AndrewSisley AndrewSisley commented Apr 13, 2023

Relevant issue(s)

Resolves #1351 #1349
May also resolve much of the current test flakiness, but there may be other underlying issues so I'll leave those open for a bit. It does not resolve all of the flakiness and I have seen one P2P local failure on this branch (logged in #1338 )

Description

Modifies the simpleChannel to guarantee the processing order of stuff.

The previous solution did not do this, and allowed new subscriptions to pick up database events that had previously occurred but had not yet been processed by handleChannel.

A common scenario for how this can result in a test failure is documented in #1351

Also includes a fix for #1349 as I believe it occurs in the same part of the codebase, for similar reasons, and there is no point putting up a second, dependant review for it.

@AndrewSisley AndrewSisley added bug Something isn't working area/db-system Related to the core system related components of the DB action/no-benchmark Skips the action that runs the benchmark. labels Apr 13, 2023
@AndrewSisley AndrewSisley added this to the DefraDB v0.5.1 milestone Apr 13, 2023
@AndrewSisley AndrewSisley requested a review from a team April 13, 2023 21:48
@AndrewSisley AndrewSisley self-assigned this Apr 13, 2023
This fixes #1349 where events are attempted to be handled after database close
@AndrewSisley AndrewSisley force-pushed the sisley/fix/I1351-event-order-of-ops branch from 2673ea4 to 5e69288 Compare April 13, 2023 21:51
@codecov
Copy link

codecov bot commented Apr 13, 2023

Codecov Report

Merging #1352 (55b61c7) into develop (a5cf397) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1352      +/-   ##
===========================================
- Coverage    70.66%   70.62%   -0.05%     
===========================================
  Files          184      184              
  Lines        17819    17818       -1     
===========================================
- Hits         12592    12584       -8     
- Misses        4275     4279       +4     
- Partials       952      955       +3     
Impacted Files Coverage Δ
events/events.go 100.00% <100.00%> (ø)
events/simple.go 91.04% <100.00%> (-0.14%) ⬇️

... and 5 files with indirect coverage changes

}

// NewSimpleChannel creates a new simpleChannel with the given subscriberBufferSize and
type subscribeCommand[T any] Subscription[T]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Fred suggested dropping the command suffix for these, or shortening it to cmd. I have a minor preference for the current names (I like it being clear that they are commands, and not an e.g. subscribe type, I also prefer the more verbose name).

Happy to change this though if others want, or if Fred's preference is/becomes stronger than I think it is :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be against dropping the suffix because at the moment it clearly shows that these types belong to a single concept. Shortening would be fine though especially because cmd is well-known. But my personal preference is to keep command

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer to keep it the way it is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will keep as-is then :)

Language rules prevented the same for publishCommand as the generic is not allowed (would be ).
This is a change from the prior solutino, as the (un)subscribe and event channels were separate.
@AndrewSisley AndrewSisley force-pushed the sisley/fix/I1351-event-order-of-ops branch from e169c05 to 55b61c7 Compare April 14, 2023 01:09
Copy link
Contributor

@islamaliev islamaliev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Tested this quite a few times again, and can confirm it is significantly less flakey! Even had a pass on my machine that would never pass before.

@AndrewSisley
Copy link
Contributor Author

LGTM! Tested this quite a few times again, and can confirm it is significantly less flakey! Even had a pass on my machine that would never pass before.

Really glad to hear - thanks again for your help with this one :)

@AndrewSisley AndrewSisley merged commit 5b08fa0 into develop Apr 14, 2023
12 checks passed
@AndrewSisley AndrewSisley deleted the sisley/fix/I1351-event-order-of-ops branch April 14, 2023 13:55
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
* Guarantee event processing order

* Make Close synchronous

This fixes sourcenetwork#1349 where events are attempted to be handled after database close
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/no-benchmark Skips the action that runs the benchmark. area/db-system Related to the core system related components of the DB bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

simpleChannel does not guarantee order of execution
3 participants