Skip to content

Conversation

@conradludgate
Copy link
Contributor

@conradludgate conradludgate commented May 16, 2025

See #11942

Idea:

  • if connections are short lived, they can get enqueued and then also remove themselves later if they never made it to redis. This reduces the load on the queue.
  • short lived connections (<10m, most?) will only issue 1 command, we remove the delete command and rely on ttl.
  • we can enqueue as many commands as we want, as we can always cancel the enqueue, thanks to the intrusive linked lists BTreeMap.

@github-actions
Copy link

github-actions bot commented May 16, 2025

8492 tests run: 7911 passed, 0 failed, 581 skipped (full report)


Code coverage* (full report)

  • functions: 32.0% (9033 of 28222 functions)
  • lines: 48.4% (80362 of 166198 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
02cd4c1 at 2025-06-20T11:31:58.011Z :recycle:

@conradludgate conradludgate force-pushed the conrad/batch-processing branch 2 times, most recently from 882d238 to f808ba9 Compare May 26, 2025 15:12
@conradludgate conradludgate changed the title [proxy] experiments with batch processing overhauls [proxy] replace the batch cancellation queue, shorten the TTL for cancel keys May 26, 2025
@conradludgate conradludgate force-pushed the conrad/batch-processing branch 2 times, most recently from 3247439 to 1e0b22b Compare June 10, 2025 14:12
@conradludgate conradludgate changed the base branch from main to conrad/minor-optimisations-for-cancellation June 10, 2025 14:12
@conradludgate conradludgate changed the base branch from conrad/minor-optimisations-for-cancellation to main June 10, 2025 14:13
@conradludgate conradludgate force-pushed the conrad/batch-processing branch 3 times, most recently from d1ebb55 to 3370e8c Compare June 10, 2025 20:40
@conradludgate conradludgate marked this pull request as ready for review June 10, 2025 20:41
@conradludgate conradludgate requested a review from a team as a code owner June 10, 2025 20:41
@conradludgate conradludgate requested a review from awarus June 10, 2025 20:41
@conradludgate conradludgate force-pushed the conrad/batch-processing branch from b509982 to 7c469b3 Compare June 11, 2025 05:43
@conradludgate conradludgate force-pushed the conrad/batch-processing branch from 2c5e757 to 02cd4c1 Compare June 20, 2025 10:22
Copy link
Contributor

@awarus awarus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks way better than we had before. Nevertheless, I think we should run it in staging with some prod-like load before releasing

@conradludgate
Copy link
Contributor Author

Yeah, I at least ran it in staging to verify the cancellation works after 10 minutes (it does work). We would need to hit it with high connection request rate to verify (I can give that a go in a moment)

@conradludgate
Copy link
Contributor Author

I'll merge now so it stays in staging. I'll run some tests (useful now that we can edit our rate limits :D)

@conradludgate conradludgate added this pull request to the merge queue Jun 20, 2025
Merged via the queue into main with commit a298d2c Jun 20, 2025
103 checks passed
@conradludgate conradludgate deleted the conrad/batch-processing branch June 20, 2025 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants