Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: batched reassign and requeue #701

Closed
wants to merge 6 commits into from
Closed

Conversation

grutt
Copy link
Contributor

@grutt grutt commented Jul 8, 2024

Description

Addresses db load under high throughput

Type of change

  • Bug fix (non-breaking change which fixes an issue)

What's Changed

  • batches assignment of step runs
  • limits total number of reassigns per poll

@grutt grutt requested a review from abelanger5 July 8, 2024 20:15
Copy link

vercel bot commented Jul 8, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
hatchet-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 9, 2024 11:51am


stepRunId := sqlchelpers.UUIDToStr(stepRunCp.StepRun.ID)
// wrap in func to avoid leaking spans
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not sure if this is needed

g.Go(func() error {
ctx, span := telemetry.NewSpan(ctx, "handle-step-run-requeue-step-run")
defer span.End()
scheduleCtx, cancel := context.WithTimeout(ctx, 1*time.Second)
Copy link
Contributor Author

@grutt grutt Jul 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about this. it should be fine considering requeueAfter is set to now + 4s and initial context takes up 1 second and sets sr into a state so they should not be re-re-assigned. but writing 100 sr updates maybe shouldnt take more than 1s?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant