Aggressive image caching breaks image badges #224

Closed
rivera-ernesto opened this Issue Oct 8, 2013 · 47 comments

Projects

None yet
@rivera-ernesto

Seems like the way README images are cached by GitHub is too aggressive.

For instance its braking the Cocoapods' badges like the one from this (README)[https://github.com/CyberAgent/iOS-NBUKit/blob/master/README.md] by replacing the perfectly functional image URL:

http://cocoapod-badges.herokuapp.com/v/NBUKit/badge.png
Correct
(The image above may be also wrongly cached!)

By this
https://github-camo.global.ssl.fastly.net/4d850e7dd43feadf363ed44e160b2296e6b21c92/687474703a2f2f636f636f61706f642d6261646765732e6865726f6b756170702e636f6d2f762f4e42554b69742f62616467652e706e67
Wrong

Seems like using https instead of http works.
https://cocoapod-badges.herokuapp.com/v/NBUKit/badge.png
Now https is also broken.
Correct

@bkeepers
Member

We have to proxy HTTP requests to avoid mixed content warnings, and therefore also cache the content to avoid excessive load. The HTTPS versions do not get proxied.

Update: The HTTPS versions also get proxied now. Assets must include Cache-Control: no-cache and ETag headers. If a badge is not updating, then it means they are not properly setting these headers. See my comment below for more info.

@bkeepers bkeepers closed this Jan 28, 2014
@rivera-ernesto

Then maybe it should be stated in the documentation somewhere. Maybe https://github.com/fletcher/MultiMarkdown/blob/master/Documentation/Markdown%20Syntax.md?

Thanks for the explanation!

@jmalloc
jmalloc commented Jan 29, 2014

HTTPS images are cached now ... https://github.com/blog/1766-proxying-user-images I can certainly understand the need. Hopefully the cache times are kept low so that status badges aren't completely useless ;)

@rivera-ernesto

Ouch

@bkeepers
Member
bkeepers commented Feb 3, 2014

@josh Can you speak into the cache ttl and how it effects build status images?

@josh
Member
josh commented Feb 3, 2014

The CDN caching respects HTTP Cache-Control. Set it accordingly.

@tomchentw

Hmm... badges are cached now, one can see the coverage on Github are outdated compared to NPM's:

https://github.com/tomchentw/ng-fire-alarm
https://npmjs.org/package/ng-fire-alarm

Any ideas?


Updated: 2014/02/18

The badge of coveralls.io still doesn't work.
#224 (comment)

@bradrydzewski bradrydzewski referenced this issue in drone/drone Feb 10, 2014
Closed

Cache-Control for Badges #41

@tomchentw

It's annoying, I tryed to recreate my coverall.io project. After a success build, it still references to the wrong badge. WTF?

@h12w
h12w commented Feb 16, 2014

I also met the same issue of obsolete coveralls.io badge. The update may take several hours.
Also:
rvagg/nodei.co#9

@tbranyen

Not sure why this is closed. It's still a huge problem. If a service wants to provide a badge, why not leave it up to them to provide the CDN?

As it stands the badges are fairly useless now on projects I maintain.

@rivera-ernesto

Also a problem when updating screenshots or just any image.

@bkeepers
Member

This should not be a problem as long as the external service sets the proper headers. For example, here are the relevant headers in a response from Travis CI.

curl -I https://api.travis-ci.org/github/linguist.png
HTTP/1.1 200 OK
Cache-Control: no-cache
Etag: "cbe59136ae412f9b0a95b9a0d2ae4d30"
Expires: Mon, 17 Feb 2014 15:21:20 GMT
Last-Modified: Mon, 17 Feb 2014 07:19:50 GMT
Pragma: no-cache

coveralls.io is setting Cache-Control. That should be enough (I'll look into why that's not). If anyone has connections at coveralls.io, you might want to suggest that they also Expires, and if they can support it, Last-Modified and Etag.

@bkeepers
Member

@tbranyen, @rivera-ernesto what services are you seeing issues with? Let me know and I'll look into why they're not working.

@h12w
h12w commented Feb 17, 2014

From my test, coveralls.io redirects to AWS and does not set Cache-Control:

curl -L -I https://coveralls.io/repos/hailiang/gspec/badge.png?branch=master

And travis-ci seems to be correct:

curl -L -I https://travis-ci.org/hailiang/gspec.png?branch=master

Hǎiliàng

@topaz2
topaz2 commented Feb 17, 2014

coveralls.io DOES affect on this issue.

Here's live sample.
First one shows 40% badge, while second one shows 100% badge.

https://github.com/NetCommons3/NetCommons3
https://coveralls.io/r/NetCommons3/NetCommons3?branch=master

@tbranyen

@bkeepers Travis-CI and Coveralls are both displayed correctly when I hit the link they give me. They are still not updated on my repo; its been around 12 hours.

@bkeepers bkeepers referenced this issue in lemurheavy/coveralls-public Feb 17, 2014
Closed

Set `Cache-Control` or `Expires` on S3 assets #220

@bkeepers
Member

@hailiang Thanks, you are right. I opened an issue with coveralls: lemurheavy/coveralls-public#220

@tbranyen I'm not seeing any issues with Travis. Could you contact GitHub Support with more info on your issue so we can look into it?

@bkeepers
Member

Related to travis-ci/travis-ci#1970. We are investigating the issue with Travis CI.

/cc @atmos

@h12w
h12w commented Feb 18, 2014

Still it does not work.

I tested it again, but it still does not show the new coverage badge, even
though coveralls.io now has Cache-Control: no-cache, Last-Modified and ETag.

curl -L -I https://coveralls.io/repos/hailiang/gspec/badge.png?branch=master

@bkeepers
Member

@hailiang It is still cached from before. I'll post back here when we get it flushed from the cache.

@atmos
atmos commented Feb 18, 2014

I busted the caches earlier today. You should be all good again.

@tomchentw

@atmos ng-fire-alarm works now! Thanks!

@backspace backspace added a commit to backspace/BitHub that referenced this issue Feb 26, 2014
@backspace backspace Serve current commit price with no caching
As complained about here:
http://www.reddit.com/r/Bitcoin/comments/1yu6l7/someone_just_sent_5000_in_btc_to_open/cfnsp03

The need to explicitly forbid caching is explained here:
github/markup#224

FREEBIE
ea0811f
@dulaccc
dulaccc commented Mar 12, 2014

I've got the issue with CircleCI badges, even though they are using Cache-Control: no-cache

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-length: 0
Content-Type: image/png
Date: Wed, 12 Mar 2014 20:49:50 GMT
Pragma: no-cache
Server: nginx/1.2.6 (Ubuntu)
Set-Cookie: ring-session=PNMPA224cdsdkB4XCmFppPWHNu2Yit%2FCgrNZo4U%2B0kw%3D--YPFgD5o0emxTdDeSb0jga8TCZbsz0pDN9teZgxR%2Bmvg%3D;Secure;Max-Age=31536000;Expires=Thu, 12 Mar 2015 18:15:04 +0000;HttpOnly;Path=/
X-Circleci-Identity: i-bb818995
X-Route: /gh/:username/:project/tree/:branch.png
Connection: keep-alive
@weilu
weilu commented Mar 14, 2014

Our Testling badge doesn't seem to update: https://github.com/bitcoinjs/bitcoinjs-lib. I checked to be sure that testling has cache-control set.

curl -L -I https://ci.testling.com/bitcoinjs/bitcoinjs-lib.png
HTTP/1.1 200 OK
content-type: image/png
cache-control: no-cache, no-store, must-revalidate
Date: Fri, 14 Mar 2014 04:11:27 GMT
Connection: keep-alive
@weilu weilu referenced this issue in bitcoinjs/bitcoinjs-lib Mar 14, 2014
Closed

Add testling to show browser support status #48

@bkeepers
Member

The assets may also need to include either an ETag or Expires header. Fastly's docs on Cache-Control say that no-cache means "re-validate before serving this content", but it doesn't specify what it does to "re-validate". I'm guessing it is re-validating, but there's no indication that the asset has changed, so it continues to serves the cached asset.

An ETag would be the biggest win, since it would guarantee that the cache gets refreshed when it changes, but still saves bandwidth.

I will try to get in touch with someone at CircleCI and testling.com.

@Sharparam

There seems to be issues with Gemnasium badges too, they set Cache-Control: private, unsure what that means to Fastly. They do set ETag and Last-Modified though.

Example:

$ curl -L -I https://gemnasium.com/RocketFuelLeak/RocketFuelLeakSite.png
HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Sun, 16 Mar 2014 14:37:02 GMT
Content-Type: image/png
Content-Length: 2823
Connection: keep-alive
Content-Disposition: inline; filename="green.png"
Cache-Control: private
Etag: "77726beb4af9335c974dc8cabfed27fb"
Last-Modified: Sat, 15 Mar 2014 21:15:08 GMT
Accept-Ranges: bytes
@rozza rozza added a commit to rozza/embeddable-build-status-plugin that referenced this issue Mar 28, 2014
@rozza rozza Added Cache-Control: no-cache
To prevent github caching of images - see github/markup#224
6a97c7f
@rozza rozza referenced this issue in jenkinsci/embeddable-build-status-plugin Mar 28, 2014
Merged

Added Cache-Control: no-cache - refs [JENKINS-17028] #6

@skidding
skidding commented Apr 1, 2014

The badge is cached wrongly and is no longer visible

screenshot 2014-04-01 13 58 27

and this is the img tag

<img src="https://camo.githubusercontent.com/33cb2bc8d83dfc59c434ec91bc56f8c0d8a9573f/68747470733a2f2f7472617669732d63692e6f72672f7562657256552f677269642e7376673f6272616e63683d6d6173746572"
     alt="Build Status"
     data-canonical-src="https://travis-ci.org/uberVU/grid.svg?branch=master"
     style="max-width:100%;">

If I change the image src to some other branch it works the first time and after refresh it is cached wrongly again and shows the missing rectangle.

@bkeepers
Member
bkeepers commented Apr 2, 2014

@skidding are you seeing this on https://github.com/uberVU/grid? It is working for me.

ubervu_grid

@skidding
skidding commented Apr 2, 2014

Started working after a few hours, thanks

@wking wking referenced this issue in ReactionMechanismGenerator/RMG-Py Apr 2, 2014
Merged

Updates to the readme #202

@eahutchins

@bkeepers is there any way to tell which branch the browser page was looking at? Prior to the caching getting turned on the referrer: had the currently viewed branch after the repo name.

@xflr6 xflr6 referenced this issue in badges/pypipins May 3, 2014
Closed

Badges do not update with GitHubs caching #15

@gordonmcshane gordonmcshane added a commit to gordonmcshane/embeddable-build-status-plugin that referenced this issue May 4, 2014
@rozza @gordonmcshane rozza + gordonmcshane Added Cache-Control: no-cache
To prevent github caching of images - see github/markup#224
26a9768
@jc00ke jc00ke added a commit to jc00ke/travis-build-status that referenced this issue May 18, 2014
@jc00ke jc00ke Fix headers
Serving svg now, plus github/markup#224 for broken caching.
efd20ac
@jc00ke
jc00ke commented May 18, 2014

This image caching is busting rubinius/rubinius#2006 and rubinius/rubinius#2121.

Before camo was caching images, jc00ke/travis-build-status could show up-to-date build statuses from Travis. I was doing this by using a redirect. I tried adding Cache-Control: no-cache but it doesn't look like that'll work for a redirect.

I attempted to "bust" the cache by adding an extra URL param to https://travis-build-status.herokuapp.com/?owner-name=wenzowski&repo-name=bcrypt-ruby&ruby-engine=rbx-18mode&x=1 but that didn't help either.

test

Any suggestions? I'd love to get this working again. Thanks!

@atmos
atmos commented May 18, 2014

curl -X PURGE camoified-url

It looks like your location header in the redirect is missing a hostname. It's probably worth setting the full URL.

@jc00ke
jc00ke commented May 18, 2014

@atmos it had the hostname but not a protocol. Adding https fixed it.

@alexzorin alexzorin referenced this issue in drone/drone Jun 7, 2014
Closed

Badge caching #257

@FabianFrank

Very annoying issue, it is breaking our build badges too.

@bkeepers
Member
bkeepers commented Jul 6, 2014

@FabianFrank what service are you having problems with?

@FabianFrank

@bkeepers A teamcity instance that's accessible via http (not https). The instance is behind a VPN and since Github is not in that VPN (obviously) it fails to fetch the badge image. This proxying no matter what is really not a good feature, at least it should be possible to opt out.

@Stasik0
Stasik0 commented Jul 7, 2014

our coveralls coveralls.io badges are also cached and broken since yesterday https://github.com/acplt/open62541/blob/master/README.md

@dbrgn
dbrgn commented Jul 7, 2014

Same issue as @FabianFrank here. Internal build server. The badge should be visible internally, it does not matter whether it breaks from the outside or not. Please provide a way to turn off the caching.

@bogdanRada

for me it works if i have the headers of the Cache-Control and the Expires .
you can check it here : https://github.com/bogdanRada/ruby-gem-downloads-badge/blob/master/README.md

But it only works if all links are using HTTP.
Using links with HTTPS doesn't work for some reason .

@bogdanRada

actually for me works with HTTPS also, i just needed to change the urls , since they were already cached and it didn't saw the new headers i added. But after changing them it works

@bkeepers
Member
bkeepers commented Jul 9, 2014

I'm looking into how we could accommodate assets that are behind a VPN. I think the ideal solution would be that the CDN would just do a redirect on assets that it can't access, but I don't know if that is feasible or what the ramifications are.

@FabianFrank

@bkeepers I guess you mean that Github would do a redirect to the resource if it can't fetch it?

@bkeepers
Member
bkeepers commented Jul 9, 2014

@FabianFrank yeah that's what I meant, corrected comment above.

@bkeepers
Member
bkeepers commented Jul 9, 2014

Sorry, I looked into it and handling assets behind a VPN is not going to work right now. CDNs don't really support the idea of redirecting you to a private resource. Someday we might investigate giving repository owners the ability to disable image caching, but it doesn't make sense for us to work on right now.

If this affects you, I would encourage you to contact support@github.com so we can track it and have a sense of how many people this impacts.

@bkeepers bkeepers locked and limited conversation to collaborators Jul 9, 2014
@bkeepers
Member
bkeepers commented Jul 9, 2014

We're pretty confident that the image caching works. If you're having issues with something like a CI badge, make sure the image has the Cache-Control: no-cache header, and either Expires, Last-Modified or Etag. See the Fastly documentation for more info.

If you're still having issues, contact support@github.com

@azu azu added a commit to azu/voting-badge that referenced this issue Jul 23, 2014
@azu azu feat(badge): set header 53ef68a
@elliottsj elliottsj added a commit to elliottsj/slackin that referenced this issue Apr 27, 2015
@elliottsj elliottsj Add Cache-Control: no-cache to /badge.svg
Hopefully this prevents GitHub's agressive caching, i.e.:
  - github/markup#224 (comment)
e3baf77
@larshp larshp referenced this issue in sbcgua/abap_data_parser Aug 28, 2016
@sbcgua sbcgua Added abaplint banner, minor readme fixes 555e70b
@guypod guypod referenced this issue in igrigorik/ga-beacon Sep 8, 2016
@igrigorik igrigorik remove github example from readme
GitHub proxies both HTTP and HTTPS resources, which means the beacon does not receive all the hits. Hence, removing the GitHub example. That said, the good news is that GitHub now offers own analytics: https://github.com/blog/1672-introducing-github-traffic-analytics
6acd862
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.