Rivet needs a caching forward proxy for many internal components.
- Self-hostable
- High performance
- Pull-through cache
- Serve cold requests while caching at the same time
- Instead of downloading the file to the cache then serving, effectively doubling the download time
- Cache hierarchy for sharded hashes
- Hash based routing can be used if not supported natively
Varnish
TODO
Squid does not support serving files at the same time as caching them.
Squid is also notoriously slow. Its architecture directly led to the development of Squid.
NGINX has a caching module, but you must pay for NGINX Plus to use it. NGINX Plus starts at $3,675 per module. This is a non-starter for an open source project and is far too expensive for an early stage startup.
TODO
S3 global replication provides many of the same performance benefits that we get out of Traffic Server.
However, it requires us to replicate our entire S3 buckets which is incredibly expensive, considering the amount of Docker images & CDN assets we store.
The requests going straight to S3 will be very expensive, we'll be billed per request. Traffic Server is within the same datacenter and costs nothing to request a prewarmed file.
- Traffic Server often fails in non-obvious ways without logging, you need to know what you're doing before using it
- While we don't need to Dockerize it at the moment, Traffic Server has a slew of bugs inside of Docker
- Development is sluggish