Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Webhooks randomly fail with 408 timeout #5470
I've setup a webhook between my server and a server at a remote location, I'm getting random failures. Some webhooks go through fine, and others die for some strange reason.
Gitea claims the following in the webhook log:
...but the remote server's nginx claims that it's Gitea's fault for not sending a complete request in time:
The remote server has a good-quality fibre line. The git server has a stable 100mb/sec connection from KimSufi / OVH. Why the timeout?
Annoyingly, it always works if I hit "test delivery" - it's only genuine push events that are causing problems.
Edit: Nope, it doesn't always work with the "test delivery" button. I've just got this:
....clicking the "test delivery" button again causes the next one to succeed though.
I'm getting a similar thing sort of randomly
If I try a Test Delivery, it usually works fine, although that could be because the first invocation already did some time-consuming downloads - and the second invocation can complete much quicker. I'm using the same webhook @sbrl is using, although not behind nginx. The script it invokes can take maybe 5-15 seconds, perhaps even longer. It'd be nice to know what the timeout was, and even better if it could be configurable.
I think the timeout is already configurable, @barryp. It's this in
[webhook] DELIVER_TIMEOUT = 60
....60 is the value I have set. If I try the "test" delivery, it does work fine some of the time. If I try it manually with
@lunny Oh dear. That's awkward. Do you have an email address I can send this packet capture to, if you think it'd help?
If further details are needed, I could try doing it over regular http. If you've got any other suggestions as to what could potentially be the cause, I'd love to hear it and I'll investigate.
I'm currently investigating using the git post-receive hook instead and writing a bash script as a workaround.