GitHub response caching #203
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…reviously received responses, thus allowing GitHub to return 304 Not Modified. This has a huge advantage for instances running multiple syncers, because cached responses don't count towards the GitHub rate limit, making it much easier to run more syncers and not getting closer to the limit. Now we'll only get a real response once something has changed on the server.
…ws for Buildasaur to handle many more syncers with more GitHub data. Now, Buildasaur caches all responses in memory and uses conditional requests to find out what has changed and only downloads that. Not only does this reduce the total bandwidth by more than 95%, it also doesn't count towards to GitHub rate limit, making Buildasaur much more scalable.
|
The interesting work is actually done here: buildasaurs/BuildaUtils@019461a |
Result of Integration 1Duration: 4 minutes and 59 seconds |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added a smart URL response cache, which sends ETag in a header from previously received responses, thus allowing GitHub to return 304 Not Modified. This has a huge advantage for instances running multiple syncers, because cached responses don't count towards the GitHub rate limit, making it much easier to run more syncers and not getting closer to the limit. Now we'll only get a real response once something has changed on the server.
Closes #62.