Docs don't load correctly during the "reload window"

[snoyberg]

Here's how loading up Haddocks works:

1. Request comes in
2. If docs are unavailable, start unpacking them and sending a "refresh" response with an HTML page until available
3. Serve the file

The problem is that, with horizontal scaling, one server may have the docs unpacked while another does not. In that situation, the HTML may load, but the CSS will get a "refresh" HTML page, which is meaningless. We should come up with a better solution.

One possibility I thought of was sending a redirect with a 100ms delay to the same URL (maybe with a different query string) so that there's an opportunity to load the content from a different server.

Pinging @manny-fp (we discussed this together) and @chrisdone (maybe you have some ideas?).

[borsboom]

Couple of ideas:

• Use sticky sessions. Rather avoid this if possible, it makes scaling work less well and won't be 100% reliable anyway.
• Keep the haddock cache in Redis instead of on the local system. That way all hosts will share the cache so this problem goes away.

[snoyberg]

I'm not sure if Redis is the right place for this, as there's a lot of content, and if the Redis server goes down, we end up in the same problem we're in now. I could picture storing all of the content on S3, and if we're adventurous, we could use the same hash-deduplication technique and store link info in the database.

I agree on avoiding sticky sessions too.

Due to the relative simplicity of it, I'm tempted to try out the redirect approach first, and if that doesn't give desired results, we can figure out some kind of cache.

[snoyberg]

Will be fixed by the move to S3, which will be released soon (probably beginning of next week).