Skip to content

Commit for every returned task is processed#837

Merged
bennybp merged 1 commit intomainfrom
queue_commit
May 20, 2024
Merged

Commit for every returned task is processed#837
bennybp merged 1 commit intomainfrom
queue_commit

Conversation

@bennybp
Copy link
Copy Markdown
Contributor

@bennybp bennybp commented May 20, 2024

Description

Previously, nested transactions were used when a manager returns tasks. That way, there would only be one big COMMIT at the end when all returned tasks had been processed.

This leads to some deadlocks between services and the task queue. So now we will commit after each returned record has been processed.

Additional disk I/O should be small, and is otherwise ameliorated postgres-side by disabling synchronous commits (which I do on production servers).

Changelog description

Commit for every returned task

Status

  • Code base linted
  • Ready to go

@bennybp bennybp merged commit b8018e6 into main May 20, 2024
@bennybp bennybp deleted the queue_commit branch May 20, 2024 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant