Site broken: all served files get truncated

[stomar]

Almost all served content seems to get truncated. This affects pages (html), RSS feeds, images, ...

This doesn't seem to be caused by a size limit; even very small files like the hamburger logo (109 bytes) are affected.

Generating and serving the site locally works fine.

This should be the common reason for #1053, #1069, #1070.

cc @hsbt

[stomar]

Regarding the timeline:

• Heroku release v285 (06b30b1, deployed 2015/04/15 23:26:54) seems to have been ok, at least regarding problems with the ruby pattern image
• Heroku release v286 (65ba0b7) showed problems with above mentioned image and was rolled back to v285
• Next deploy: v290 (08e7f27, at 2015/05/03 11:16:42); this matches the statement that RSS feeds started being broken at about May 2nd; truncated pages were observed at May 5th

So the problem seems to have been introduced between these commits:

• 06b30b1 Thu Apr 16 15:18:04 2015 +0900
• 65ba0b7 Fri Apr 17 18:12:37 2015 +0000

[stomar]

@hsbt Between these commits was the switch to Jekyll 2:

9b90c45 Thu Apr 16 15:48:20 2015 +0900

😢

Update: Jekyll 2 itself is not the problem, see below.

[stomar]

Update: It's an issue with rack-jekyll 0.3.5 !!!

Serving locally with jekyll serve does work, but actually that's not what the server does.
So I tried with rackup config.ru and got internal server errors, e.g.

ERROR Rack::Lint::LintError: Content-Length header was 107, but should be 109

This was for the file images/hamburger.png; note that for this file using curl or wget results in a downloaded file with only 107 of the total 109 bytes.

So it seems that the files get served with wrong Content-Length headers, so that they are truncated by the clients.

Update: This is fixed in rack-jekyll 0.4.0 or higher (namely 0.4.1 as of now), but both versions are not compatible with Jekyll 2. The current master branch of rack-jekyll is.

Note: I needed to (locally) remove SSL in config.ru, since I didn't know how to get it running with SSL.

[stomar]

I tried to fix this with 06d990e, but this caused an application error, and I needed to reverted 😢

With a local dummy application it worked...

Update: It seems the development version of rack-jekyll always regenerates the site; this crashes on Heroku. I already opened an issue there.

[stomar]

To sum up: rack-jekyll is the problem.

• < 0.4.0 calculates wrong Content-Length headers (length instead of bytesize)
• 0.4.0 and 0.4.1 should work fine, but require Jekyll <2.0 (IMO for no particular reason)
• master works with Jekyll 2, but introduces automatic (re)generation of the site, which crashes on Heroku

[stomar]

Fixed with c01e59d: uses rack-jekyll 0.4.1, but a patched version that allows Jekyll 2.

[stomar]

@hsbt Please review.

[hsbt]

@stomar Thanks for your investigation! I confirmed you only changes jekyll version from v0.4.1. It's ok to our website.

[JuanitoFatas]

This issue can be closed right? Seems all good now! 👏

[stomar]

Fixed for the time being.

But we need to observe the future development of rack-jekyll, I would prefer switching back to the 'official' releases.