Respect cache header from custom endpoints

[muety]

I might have overlooked something, but it seems like Cache-Control headers are not properly respected for requests to custom endpoints, are they?

Requesting a badge twice using a custom endpoint, which returns Cache-Control: no-cache, resulted in only one request to that certain endpoint in my experiments.

How is does Shields implement caching in general? What resource are cached and for how long?

[chris48s]

We're not caching the response from your endpoint, but we are caching our badge images (for 5 mins). You can ask our downstream to cache it for a longer time using the cacheSeconds field, but not a shorter time (see docs at https://img.shields.io/endpoint)

If you call curl -I "https://img.shields.io/endpoint?url=<url to your endpoint> you can tell from the cf-cache-status: header whether the badge was served directly or from cache. If you get cf-cache-status: MISS then we served it direct and you should see a hit on your endpoint. If you get cf-cache-status: HIT then we served the badge from downstream cache and you won't see a hit on your endpoint.

[muety]

Thanks for the detailed explanation! I assume the fact that a badge is at minimum cached for five minutes is a measure to reduce server load / prevent denial of service attacks to a certain extent?

[chris48s]

I assume the fact that a badge is at minimum cached for five minutes is a measure to reduce server load

Exactly.

That said, we've never really had much of a discussion about why the floor is 300 seconds. We set it to that when we implemented it and it has never really been questioned, but since I'm thinking about it I'm going to write down a couple of (contradictory) thoughts.

Most of the 'native' shields badges are cached for 2 mins. Some are cached for longer. For example we set licence badges to be cached for an hour because projects switch licence very infrequently. We also cache some for a shorter time than this. For example we only cache build status badges for 30 seconds because this often changes (I think that is the shortest length we use). The lengths we've chosen here seem to be serving us pretty well. We rarely ever get anyone raising issues about badges not updating quickly enough/being cached too long and we serve about half our traffic from cache, which is great for performance/cost.

On the one hand, there is a bit of a double standard going on here. We can decide some of our badges need to be updated more frequently, but endpoint users can't. You could say we're sitting here eating people food and giving endpoint users dog food, if you were being uncharitable.

On the other hand, there is possibly a mismatch of incentives here too:
For us as maintainers of the service, there is a balancing act to perform. We want changes in the real world to be reflected soon after they happen to provide good user experience, but we also have to run a service that serves 10-20 million requests per day on donations from the community and ensure fast performance too. So as we tune those thresholds we are affected by both sides of that tradeoff.
As a user of the endpoint badge the incentives are different because you don't really have to deal with one side of that equation: For users who host their JSON endpoint on runkit, github, etc if you can set cacheSeconds=0 then your badge updates quicker and then.. someone else is hosting it, so why wouldn't you just do that? Again, that's the least charitable way of looking at it.

One of the interesting things that is happening at the moment is I am starting to see more examples of people using the endpoint badge and asking support questions about it. Looking at the stats for the last 24 hours, "endpoint" was the 10th most popular service 😮. This is really cool because it is allowing shields to have something more like a "plugin" architecture (as far as that is possible with a SaaS) where users can use the service but self-serve their own needs without us having to be gatekeepers/maintainers of everything. Maybe as that happens we might need to tweak things a bit.

I guess the point I'm meandering towards here is we picked 300 seconds when we added this a couple of years ago and nobody was really using it yet but I could be convinced that just as there are use-cases inside the shields codebase where 300 seconds is too long, maybe there are use-cases appearing in the wider community where that is also the case. For example we could change it so that cacheSeconds defaults to 300 (defaults matter) but the min is a bit lower if that enables some interesting use cases that people want.

[calebcartwright]

I'm open discussing this (perhaps at the next ops meeting), and will note #3596 as well since there could be some related concerns/decisions