Timeout after lots of API calls

[jrieke]

Summary

If I do lots (~hundreds) of requests within a few minutes, the API call sometimes times out without raising an error (i.e. it just runs indefinitely).

Details

I'm using the api.activity.list_stargazers_for_repo endpoint (didn't check if the problem persists with other endpoints). I'm not sure why this happens but I double-checked that I'm not rate-limited (using limit_cb callback + printing api.rate_limit.get() regularly). If I stop the process and start it again right after, it also works fine, so I assume it's just a temporary timeout.

Possible fix

A great workaround would already be to be able to set a manual timeout when initializing the API or calling an endpoint.

[jrieke]

For anyone looking for a (temporary) solution: I managed to monkeypatch fastcore.net so that ghapi raises an error on timeout. Before you create the GhApi object, do:

import fastcore.net
import functools
fastcore.net._opener.open = functools.partial(fastcore.net._opener.open, timeout=10)

[jph00]

This seems to be an issue with GitHub's API, so I'll close this. Thanks for documenting the workaround you identified.