Skip to content
This repository has been archived by the owner on Sep 16, 2021. It is now read-only.

Acquire and ping document event locks in parallel batch #70

Open
alechenninger opened this issue Oct 15, 2016 · 2 comments
Open

Acquire and ping document event locks in parallel batch #70

alechenninger opened this issue Oct 15, 2016 · 2 comments

Comments

@alechenninger
Copy link
Contributor

alechenninger commented Oct 15, 2016

Best would be this I think: lightblue-platform/lightblue-rest#244

But alternatively we could use a thread pool and acquire/ping all locks in parallel

@alechenninger alechenninger changed the title Acquire locks in parallel batch Acquire and ping locks in parallel batch Oct 15, 2016
@alechenninger
Copy link
Contributor Author

alechenninger commented Oct 18, 2016

For an idea of impact, in a non-prod environment, the time spent for each notification was:

  • 18ms to find it
  • 240ms acquiring, pinging locks
  • ?ms doing in memory logic (examing the notification)
  • 140ms to mark it processed

By parallelizing lock acquisition and pinging, in a batch of 100 notifications, we could reduce the time per notification by about 54% (to 182ms in this case).

@alechenninger
Copy link
Contributor Author

alechenninger commented Oct 18, 2016

Another thought: do we even need locks?

Couldn't we just do:

  1. Find notifications
  2. Update to status processing if current ... here we will catch if another thread is already processing the notification. We have to do this either way.
  3. Any not updated by 2 are dropped since we assume another thread is processing it.

@alechenninger alechenninger changed the title Acquire and ping locks in parallel batch Acquire and ping document event locks in parallel batch Nov 20, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant