Request for gzip Compression for HTML / CSS / JS

[ghost]

I'd like to see optional gzip compression in Bottle, mainly because I have some bigger static JavaScripts in my project. If you reject this request, maybe you could give me a hint where it would be best to add this functionality?

Thanks in advance.

Thomas

[defnull]

There is a paste middleware that does this, but it is quite limited:
http://pythonpaste.org/modules/gzipper.html#paste.gzipper.middleware

You are not the only one that requested such a feature. I'll try to add this to the core.

[royshan]

recently, we released a wsgi gzip middleware. it only compresses selected types of contents. also it is small and easy to configure. check it out at http://code.google.com/p/ibkon-wsgi-gzip-middleware/ .

[sc68cal]

I know this is quite an old issue, but I think that GZIP is out of the purview of Bottle, and should be handled by middleware, similar to how authentication is handled by a middleware.

In fact, wouldn't Flup be well suited for this?

EDIT: Looks like those modules were removed.

[defnull]

EDIT: Looks like those modules were removed.

For a reason, I suppose. It is not that easy to get compression right. You need to:

• Compress on the fly and be fast doing so.
• Do not compress for browsers that don't support it.
• Do not compress files that are compressed already (images, videos).
• Do not compress dynamic files.
• Support two differed compression algorithms (gzip and deflate).
• Cache compressed files that don't change often.
• De-validate the cache if one of the files changed anyway.
• Make sure the cache does not get to big.
• Do not cache small files because a disc-access would take longer than on-the-fly compression.

And many other edge cases I don't remember. I currently do not now a WSGI middleware that does all this in a satisfying way. It is usually better solved by a web-server module (lighttpd+mod_compress or Apache+mod_deflate).

Since 512048b, static_file adds a Content-Encoding header if the file appears to be compressed. For example, a big_script.js.gz file is sent with Content-Type: application/javascript and Content-Encoding: gzip headers.

If you only want to speed up some big CSS and JavaScript files, you could compress them manually and add some logic to your static-file route:

@route('/static/:filename#.*#')
def static(filename):
if 'gzip' in request.headers.get('Accept-Encoding'):
out = static_file(filename+'.gz', '/path/to/statics/')
if not isinstance(out, HTTPError):
return out
return static_file(filename, '/path/to/statics/')

This searches for a file with a '.gz' extension and returns that if the browser supports it. If not, the uncompressed file is returned.

If done right, something like this could be added to the core I suppose. Pior [1] played with that Idea some time ago. His branch is still interesting.

[1] https://github.com/pior/bottle

[sc68cal]

    For a reason, I suppose. It is not that easy to get compression right. You need to:
    ...
    And many other edge cases I don't remember. I currently do not now a WSGI middleware that does 
    all this in a satisfying way. 
    It is usually better solved by a web-server module (lighttpd+mod_compress or Apache+mod_deflate).

Agreed. I think re-inventing the wheel to solve all these edge cases is a waste of resources, it would be much better to document the advantages and disadvantages to WSGI middleware, and then also mention web-server modules that implement it better, while linking some middleware projects in the docs. I'd be happy to implement that.

[defnull]

I am awaiting your pull request :) (and closing this issue)

[sc68cal]

Actually, could you reopen it? That way when you merge the pull request you can put in a "closes #92" in the message and it'll link up the history.

[sc68cal]

Sorry for the noise, rebased off your master then pushed again. I'm rushing too much and getting sloppy.

[sc68cal]

sighs - Ok - this time issue_92 is a clean branch you can merge. Had a commit that belongs in a different pull request.