Skip to content

feat(machine): introduce backoff for handler timeouts - #261

Merged
pancsta merged 1 commit into
mainfrom
feat/machine/handler-backoff
Jul 8, 2025
Merged

pancsta merged 1 commit into
mainfrom
feat/machine/handler-backoff

Conversation

@pancsta

@pancsta pancsta commented Jul 8, 2025

Copy link
Copy Markdown
Owner

Fault tolerancy now works well under a heavy load with handlers timing out:

  • handler is started
  • timeout is reached (100ms)
  • handle has to return within HandlerDeadline (10s)
  • as early as Event.IsValid() returns true
  • transition is canceled anyway
  • in case of no return, the machine goes into a backoff state
  • all the queue is flushed down the drain
  • machine cancels any mutations for the period of HandlerBackoff (3s)
  • new mutations run on a new event loop

This achieves goroutine cancellation based on probabilities (similar to hash collisions), and can be adjusted on per-machine and per-usecase basis.

@pancsta
pancsta merged commit 70eb2be into main Jul 8, 2025
@pancsta
pancsta deleted the feat/machine/handler-backoff branch July 8, 2025 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(machine): add timeout confirmations

1 participant