Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(996): Do not release database lock early for subscriptions #997

Merged
merged 1 commit into from
Mar 20, 2024

Conversation

joshua-spacetime
Copy link
Collaborator

Fixes #996.

Description of Changes

Please describe your change, mention any related tickets, and so on here.

API and ABI breaking changes

If this is an API or ABI breaking change, please apply the
corresponding GitHub label.

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.

This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do,
so that you're confident that all the changes work as expected!

  • Write a test you've completed here.
  • Write a test you want a reviewer to do here, so they can check it off when they're satisfied.

@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/out-of-order-subscription branch from a8d1ae2 to c79318e Compare March 20, 2024 01:44
@joshua-spacetime joshua-spacetime force-pushed the joshua/test/out-of-order-subscription branch from b422b08 to 690239b Compare March 20, 2024 01:49
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/out-of-order-subscription branch from c79318e to 48476ce Compare March 20, 2024 01:53
@@ -85,11 +85,9 @@ impl ModuleSubscriptions {
// This also makes it possible for `broadcast_event` to get scheduled before the subsequent part here
// but that should not pose an issue.
let mut subscriptions = self.subscriptions.write();
drop(tx);
Copy link
Contributor

@Shubham8287 Shubham8287 Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we do not need to drop(tx) early for any correctness issue, we can do it being more conserative.

dropping subscription lock should be enough as that will also not let the reducers to commit (reducer need subscription read lock to commit), so their eval_incr will also not start.

if we not drop(tx) early, reducer execution will not happen parallel to eval, though right now we are letting execution to happen but not commit (which is also fine).

At the end, it depends whether we want to be more conservative or look for bit more perfomance.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed async: Holding the tx for the lifetime of the call does keep a reducer from executing until we push onto the send queue, but that shouldn't have a noticeable impact on throughput, plus it makes testing easier. So I think we're both in favor of this from a correctness standpoint.

@joshua-spacetime joshua-spacetime added the test-with-bots Recommend to test under higher CCU label Mar 20, 2024
@joshua-spacetime joshua-spacetime linked an issue Mar 20, 2024 that may be closed by this pull request
@joshua-spacetime joshua-spacetime force-pushed the joshua/test/out-of-order-subscription branch from 690239b to ccfd964 Compare March 20, 2024 16:19
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/out-of-order-subscription branch from 48476ce to f2cc2fd Compare March 20, 2024 16:52
Base automatically changed from joshua/test/out-of-order-subscription to master March 20, 2024 17:01
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/out-of-order-subscription branch from f2cc2fd to d2d5782 Compare March 20, 2024 17:02
@joshua-spacetime joshua-spacetime added this pull request to the merge queue Mar 20, 2024
Merged via the queue into master with commit b36dda4 Mar 20, 2024
6 checks passed
@joshua-spacetime joshua-spacetime deleted the joshua/fix/out-of-order-subscription branch March 20, 2024 18:04
@joshua-spacetime joshua-spacetime mentioned this pull request Mar 20, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test-with-bots Recommend to test under higher CCU
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Subscriptions can send messages out of order
2 participants