Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rate limit improvements #1321

Merged
merged 6 commits into from
Apr 15, 2023
Merged

feat: rate limit improvements #1321

merged 6 commits into from
Apr 15, 2023

Conversation

LordOfPolls
Copy link
Contributor

@LordOfPolls LordOfPolls commented Apr 4, 2023

About

This PR rewrites all http methods to pass their params correctly, meaning rate limit buckets are properly generated and shared.

Additionally, this also rewrites the BucketLock to allow for concurrent requests to the API, rather than queuing them sequentially, improving HTTPClient throughput for large bots. Smaller bots are very unlikely to notice a difference.

Checklist

  • The pre-commit code linter has been run over all edited files to ensure the code is linted.
  • I've ensured the change(s) work on 3.8.6 and higher.
  • I have added the versionadded, versionchanged and deprecated to any new or changed user-facing function I committed.

Pull-Request specification

I've made this pull request: (check all that apply)

  • For the documentation
  • To add a new feature
  • As a general enhancement
  • As a refactor of the library/the library's code
  • To fix an existing bug
  • To resolve #ISSUENUMBER

This is:

  • A breaking change

@codecov-commenter
Copy link

codecov-commenter commented Apr 4, 2023

Codecov Report

Patch coverage: 4.89% and project coverage change: -0.03 ⚠️

Comparison is base (75713d1) 47.72% compared to head (16bde8d) 47.69%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@            Coverage Diff             @@
##              5.x    #1321      +/-   ##
==========================================
- Coverage   47.72%   47.69%   -0.03%     
==========================================
  Files         139      139              
  Lines       14282    14297      +15     
==========================================
+ Hits         6816     6819       +3     
- Misses       7466     7478      +12     
Impacted Files Coverage Δ
interactions/api/http/http_requests/channels.py 36.06% <0.00%> (ø)
interactions/api/http/http_requests/emojis.py 55.00% <0.00%> (ø)
interactions/api/http/http_requests/guild.py 30.63% <0.00%> (ø)
...nteractions/api/http/http_requests/interactions.py 41.30% <0.00%> (ø)
interactions/api/http/http_requests/members.py 43.58% <0.00%> (ø)
interactions/api/http/http_requests/messages.py 51.85% <0.00%> (ø)
interactions/api/http/http_requests/reactions.py 68.42% <0.00%> (ø)
...actions/api/http/http_requests/scheduled_events.py 65.00% <0.00%> (ø)
interactions/api/http/http_requests/stickers.py 66.66% <0.00%> (ø)
interactions/api/http/http_requests/threads.py 40.38% <0.00%> (ø)
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@LordOfPolls LordOfPolls changed the title fix: refactor all http routes to generate buckets properly feat: rate limit improvements Apr 6, 2023
@LordOfPolls
Copy link
Contributor Author

This PR is complete, however due to its scope i request a review

@LordOfPolls LordOfPolls changed the base branch from 5.x to unstable April 10, 2023 18:03
@LordOfPolls
Copy link
Contributor Author

Tested working on my end, request someone else to confirm

@LordOfPolls LordOfPolls enabled auto-merge (squash) April 11, 2023 21:51
@LordOfPolls LordOfPolls mentioned this pull request Apr 12, 2023
16 tasks
@LordOfPolls
Copy link
Contributor Author

PR is needed and becoming stale. I've tested locally and will now merge

@LordOfPolls LordOfPolls merged commit e3b9419 into unstable Apr 15, 2023
@LordOfPolls LordOfPolls deleted the bucket_fix branch April 15, 2023 15:22
LordOfPolls added a commit that referenced this pull request Apr 22, 2023
* feat: support regex component callbacks (#1332)

* feat: support regex component callbacks

* docs: document regex matching component callback

* ci: correct from checks.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* chore: switch to MIT license (#1334)

* chore: switch to MIT license

* ci: correct from checks.

* docs: update license through github 

Uses githubs own license tools so the correct license is detected

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* feat: add audit events (#1335)

* feat: add audit new mod events to audit log enum

* feat: add onboarding related events

see discord/discord-api-docs#6041

* refactor: change log level of missing value to warning (#1339)

Lets not scare people with "error"s ay?

* feat: add missing message types (#1338)

* feat: add missing message types

* feat: add deletable message types helper

* build: correct manifest

* feat: support avatar decorations (#1329)

* build: mirrored publish to d.py-interactions and interactions.py (#1336)

* style: add newline to end of manifest

* ci: fix migration issues in integration test bot

* fix: copy checks when inheriting (#1342)

Co-authored-by: Astrea49 <25420078+Astrea49@users.noreply.github.com>

* feat: cooldown upgrades (#1323)

* docs: add changelog page

* style: add EOF newline

* docs: add 5.1.0 to changelog.md

* style: remove trailing whitespace

* fix: reintroduce imports at namespace linter removed

* feat: sync improvements (#1328)

* refactor: improve readability of sync logic

* fix: improve initial sync caching

* docs: add docstring to update_command_cache

* feat: add command deletion to debug ext

* feat: rate limit improvements  (#1321)

* fix: refactor all http routes to generate buckets properly

* fix: resolve routes regex missed

* feat: allow concurrent api calls from the same bucket

* feat: restore bucketLock.locked property

* feat: further bucketlock improvements

* feat: allow bucketlock blocking to be toggled

* refactor: resolve ruff compliant

* fix: correct webhook http

* fix: wrong check condition in the component callback (#1352)

* fix: error when dispatch component callback (#1351)

* docs: adjust and fix many parts of the docs (#1353)

* docs: adjust and fix many parts of the docs

* docs: add note about event object for v4 migration

* docs: replace more instances of InteractionContext

---------

Co-authored-by: Astrea49 <25420078+Astrea49@users.noreply.github.com>

* feat: caching improvements (#1350)

* feat: add support for force fetching

* feat: track if a user object has been fetched

* feat: add force flag to client helper methods

* feat: update all cache fetch methods to have  a force param

* feat: add reset_with_key and get_cooldown_time_with_key (#1345)

* made get_cooldown_with_key async for consistency, added reset_with_key and get_cooldown_time_with_key

* ci: correct from checks.

* fixed pre-commit problems

* comment on #1345, resolved

* made Cooldown.reset_all async #1345

* bot.load => bot.load_extension

* reverted 8217f4e

* made get_cooldown_time_with_key and reset_with_key sync

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Astrea <25420078+AstreaTSS@users.noreply.github.com>
Co-authored-by: Astrea49 <25420078+Astrea49@users.noreply.github.com>
Co-authored-by: Damego <damego.dev@gmail.com>
Co-authored-by: chronosirius <73508925+chronosirius@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants