Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Separate polling & completing "halves" of WFT/activity management #180

Closed
Sushisource opened this issue Aug 24, 2021 · 0 comments · Fixed by #328
Closed

[Refactor] Separate polling & completing "halves" of WFT/activity management #180

Sushisource opened this issue Aug 24, 2021 · 0 comments · Fixed by #328
Labels
enhancement New feature or request

Comments

@Sushisource
Copy link
Member

The last round of refactoring that moved WFT management inside of workers still leaves room for improvement. Hopefully the last round of work we need to do here for likely a rather long time.

There are a bunch of synchronization concerns that would be made simpler by following the below pattern for both WFTs and Activities:

  • Polling happens in the background (this is largely already true, just need to remove limiting semaphore)
    • Locks workflow run state to make needed updates
    • Poll results are fed into a stream
  • Poller selects from stream(s) (for wft: pending, real poll results)
  • Completing locks workflow run state to make needed updates

By keeping the polling half separate from the completing half, locking (or, whatever synchronization mechanism, could be message passing) permits polling and completing to happen simultaneously per-worker, but there cannot be multiple concurrent polls. Multiple concurrent completions for different workflow runs is permitted (as it is now).

Before doing this I think it's probably worth diagramming it out, to land on a really nice final iteration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant