Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error while posting: "Too many requests" #222

Closed
tichter opened this issue Aug 7, 2018 · 13 comments
Closed

Error while posting: "Too many requests" #222

tichter opened this issue Aug 7, 2018 · 13 comments
Labels

Comments

@tichter
Copy link

@tichter tichter commented Aug 7, 2018

Hi @eduardoboucas,

I found this great framework for audio listening tasks https://github.com/deeuu/listen, using github pages and staticman. I love how easy it works and I implemented my own version of it.
I set up a task and send it to 20 people. Everything worked fine most of the time, but sometimes people and myself got an error while submitting the task via staticman. You can find a screenshot of the error below.

The problem is, that I can't reproduce the error consistently. When it occurs, after reloading the page and redoing the task, the submit works fine.
The only way I could consistently reproduce the error is when I submit the task, click the back button of the browser, in order to get back to the popup and then click submit again.

You can find a test version at https://tichter.github.io/pwlisten/. You can enter the page with the codes "test1" or "test2" (without the "). The site is still under development so you can test as much as you want and post as many requests as you want. I will reset everything before I launch the project.

Do you know where this error comes from? Is it because of the staticman api, or is it an error coming from github? Do you have any solutions for this problem?

Thank you in advance for your help,
Alex Tichter

staticmanerror

@jloisel
Copy link

@jloisel jloisel commented Aug 8, 2018

We also have this issue, despite the fact we only have a comment once every few days. That looks like an issue on staticman side (ddos attack?).

@onwingslikeeagles
Copy link

@onwingslikeeagles onwingslikeeagles commented Aug 9, 2018

Hello. My team is using Staticman and we are seeing this issue as well. The issue is happening intermittently as we are submitting data requests to the tool. Since there is a variable that denotes the next time when submitting a request would be valid, there must be a reason why the request are failing. Could a job queue be stored within Staticman that is causing the delay?

@tmadeira
Copy link

@tmadeira tmadeira commented Aug 13, 2018

+1

@leonp
Copy link

@leonp leonp commented Aug 27, 2018

Similar on my site:

{"error":{"text":"Too many requests in this time frame.","nextValidRequestDate":"2018-08-27T13:14:57.224Z"}}

Intermittent. Assumed it was spambots sending comments and the API stopping requests until a specific time. However, I was able to post a comment before nextValidRequestDate (assuming it's UTC).

Or maybe Staticman is getting too many comments and throttling?

@StephanWeinhold
Copy link

@StephanWeinhold StephanWeinhold commented Aug 27, 2018

I'm afraid it's the throttling. Because I've experienced this yesterday on a site that was live for 5 mins while sending a test comment.

@eduardoboucas
Copy link
Owner

@eduardoboucas eduardoboucas commented Aug 29, 2018

Staticman itself does not implement any throttling. I'm afraid this must be a limitation from the GitHub API, as everyone using the public instance is pinging the GitHub API using the same API key. Not sure if there's anything I can do.

@leonp
Copy link

@leonp leonp commented Aug 29, 2018

Thanks Eduardo. Guess it's best we host our own instances :-)

@jloisel
Copy link

@jloisel jloisel commented Sep 21, 2018

Thanks Eduardo. Guess it's best we host our own instances :-)

Well, hosting your own comments server pretty much cancels the benefits of a statically generated website stored on Github / S3 + Cloudfront (or equivalent).

@InsOpDe
Copy link

@InsOpDe InsOpDe commented Nov 11, 2018

@eduardoboucas Actually staticman does the throttling - with the help of ExpressBrute - you may add an option-object as the second parameter when creating a new instance of ExpressBrute in server.js:36 - see: https://github.com/AdamPflug/express-brute and if the threshold is reached, it fires the 429 response and lets you wait 15 minutes.

@casually-creative
Copy link

@casually-creative casually-creative commented Nov 12, 2018

I'm a part of the same team as @unpreceden7ed mentioned. We're still receiving this issue day to day and it has been a major blocking issue for us. Our content people who use the tool report this issue very frequently (know that they do use the tool a lot though). I think that it wouldn't be a very unwise idea to implement some sort of queue at the side of staticman? When the github api returns an error, you simply wait untill you can send again, asuring that all requests in the queue succeed eventually. Time is not an issue for us, and I think for most other implementations either. So it doesn't really matter if it happens instantly, after 2 minutes or even after 10 minutes. As long as we can be quaranteed every single request goes through with a 100% success rate, that would be incredibly helpfull! Cheers!

@aivuk
Copy link

@aivuk aivuk commented Nov 14, 2018

@michieldp, see @InsOpDe answer. I had the same problem because of the ExpressBrute and solved removing its use from https://github.com/eduardoboucas/staticman/blob/master/server.js. Just remove this.initialiseBruteforceProtection() and all the four lines with this.bruteforce.prevent from the server.js and you will not receive the error "Too many requests in this time frame" anymore.

You can see the changes here:

aivuk@0fe57eb

@casually-creative
Copy link

@casually-creative casually-creative commented Nov 15, 2018

@aivuk Thanks for your reply. Do you mind enlightening me about express brute? Not sure what it is or where it steps in exactly. We don't use express brute neither do we have a back-end nor do we host our own staticman instance..

@alexwaibel
Copy link
Collaborator

@alexwaibel alexwaibel commented Nov 9, 2019

Duplicate of #317

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet