-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle potential race condition in which Trac is notified but git fetch doesn't get the changeset #111
Comments
I think I tried that before without success, but perhaps you'll be more persistent and figure it out :-) |
I see now that the behavior is documented in known issues. |
6f84cdc has some changes that I'll be testing. |
The best practice for doing this is exponential backoff. Start with a given delay (order of magnitude: 0,5 - 1s) and at each retry, multiple the delay by a constant factor (order of magnitude: 1,5 - 2). This is easy to implement and nicer on the remote end if they're struggling. |
I'm not sure the re-try is a very good idea; GitHub seems to have pretty strict timeout policy on webhooks (they regularly mark some of ours as "timeout", even though in quite a few of the cases, Trac seems to have processed them). Maybe we should add a database table, store the webhook and trigger some asynchronous, delayed processing that can then be re-tried until it succeeds? |
Thanks for the tips. I haven't seen the issue reoccur since this last report. Running my branch with proposed changes, I also never saw log message indicating that the update had to be retried. I'll revisit the issue if it occurs again. |
I'm seeing this approximately every 2 weeks or so with trac 1.2.2. Just resending the webhook from github doesn't resolve the issue as I still get the "unknown commit" I will keep trying to understand why we get into this situation. As per previous comments, I also regularly get timeouts from the github webhooks. Github thinks I believe we should be accepting the payload from github for async processing (and returning http 202 accepted), then process what github have sent us. |
Discussed near the end of #8, I continue to see cases in which a changeset does not sync from GitHub and therefore the post-commit hook is not triggered. Running a
git fetch
andtrac-admin $env changeset added
for the changeset always triggers the update of the relevant ticket.The following is always seen in the logs when the issue occurs:
I've considered that it might work to implement a simple retry of
git remote update --prune
.The text was updated successfully, but these errors were encountered: