Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add overall api client rate limit option #19

Closed
creharmony opened this issue Apr 1, 2021 · 3 comments · Fixed by #20, #22 or #23
Closed

add overall api client rate limit option #19

creharmony opened this issue Apr 1, 2021 · 3 comments · Fixed by #20, #22 or #23
Assignees
Labels
enhancement New feature or request

Comments

@creharmony
Copy link
Owner

creharmony commented Apr 1, 2021

As Etsy limits the number of call per time window,

It would be great to have an option to calibrate client calls rate limit to never reach allowed api key rate limit.

  • option must be disabled by default
  • option must be configured by options
  • options must permit "per second" and/or "per day" windows
  • study retry after option in case of error (another issue to implement retry?)

@creharmony creharmony added the enhancement New feature or request label Apr 1, 2021
@boly38
Copy link
Collaborator

boly38 commented Apr 2, 2021

  • rate limiter without shared storage will not scale up
  • request-rate-limiter - pro : top seo, very easy to implement // con : embeds experimental-modules, enforce "module" type
  • node-rate-limiter - pro : very easy to implement // con : ?

node rate limiter

@boly38
Copy link
Collaborator

boly38 commented Apr 3, 2021

node-rate-limiter - Intensive Etsy calls with a rate limit set to 10/sec makes some etsy error as result:

You have exceeded your quota of: 10 requests per 1 second(s) for public requests.

So following this, I made some additional test :

                              duration =~ average rate
    200 queries at 10/sec |=> 19sec   =~ 10.52 /sec
    200 queries at  9/sec |=> 21.228  =~ 9.4215/sec
    200 queries at  8/sec |=> 24      =~ 8.33  /sec
    200 queries at  7/sec |=> 27.579  =~ 7.2519/sec
 

Intensive use of etsy client with a rate set to 8 or 7 per sec dont prevent etsy limit issue; maybe missing await?

Either Etsy or rate limiter is lying ;)

  • tests show that rate limiter seems slightly upper contract

TODO

creharmony pushed a commit that referenced this issue Apr 5, 2021
@boly38 boly38 self-assigned this Apr 5, 2021
creharmony pushed a commit that referenced this issue Apr 5, 2021
upd readme too
@boly38 boly38 closed this as completed in #22 Apr 5, 2021
boly38 added a commit that referenced this issue Apr 5, 2021
upd readme too
@boly38 boly38 reopened this Apr 6, 2021
@creharmony
Copy link
Owner Author

creharmony commented Apr 8, 2021

creharmony pushed a commit that referenced this issue Apr 8, 2021
@boly38 boly38 closed this as completed in #23 Apr 8, 2021
boly38 added a commit that referenced this issue Apr 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
2 participants