Request for gzip Compression for HTML / CSS / JS #92

Closed
volatilevoid opened this Issue Aug 20, 2010 · 9 comments

Projects

None yet

4 participants

@volatilevoid

I'd like to see optional gzip compression in Bottle, mainly because I have some bigger static JavaScripts in my project. If you reject this request, maybe you could give me a hint where it would be best to add this functionality?

Thanks in advance.

Thomas

@defnull
Member
defnull commented Aug 20, 2010

There is a paste middleware that does this, but it is quite limited:
http://pythonpaste.org/modules/gzipper.html#paste.gzipper.middleware

You are not the only one that requested such a feature. I'll try to add this to the core.

@royshan
royshan commented Feb 1, 2011

recently, we released a wsgi gzip middleware. it only compresses selected types of contents. also it is small and easy to configure. check it out at http://code.google.com/p/ibkon-wsgi-gzip-middleware/ .

@sc68cal
Contributor
sc68cal commented Apr 26, 2011

I know this is quite an old issue, but I think that GZIP is out of the purview of Bottle, and should be handled by middleware, similar to how authentication is handled by a middleware.

In fact, wouldn't Flup be well suited for this?

EDIT: Looks like those modules were removed.

@defnull
Member
defnull commented Apr 26, 2011

EDIT: Looks like those modules were removed.

For a reason, I suppose. It is not that easy to get compression right. You need to:

  • Compress on the fly and be fast doing so.
  • Do not compress for browsers that don't support it.
  • Do not compress files that are compressed already (images, videos).
  • Do not compress dynamic files.
  • Support two differed compression algorithms (gzip and deflate).
  • Cache compressed files that don't change often.
  • De-validate the cache if one of the files changed anyway.
  • Make sure the cache does not get to big.
  • Do not cache small files because a disc-access would take longer than on-the-fly compression.

And many other edge cases I don't remember. I currently do not now a WSGI middleware that does all this in a satisfying way. It is usually better solved by a web-server module (lighttpd+mod_compress or Apache+mod_deflate).

Since 512048b, static_file adds a Content-Encoding header if the file appears to be compressed. For example, a big_script.js.gz file is sent with Content-Type: application/javascript and Content-Encoding: gzip headers.

If you only want to speed up some big CSS and JavaScript files, you could compress them manually and add some logic to your static-file route:

@route('/static/:filename#.*#')
def static(filename):
if 'gzip' in request.headers.get('Accept-Encoding'):
out = static_file(filename+'.gz', '/path/to/statics/')
if not isinstance(out, HTTPError):
return out
return static_file(filename, '/path/to/statics/')

This searches for a file with a '.gz' extension and returns that if the browser supports it. If not, the uncompressed file is returned.

If done right, something like this could be added to the core I suppose. Pior [1] played with that Idea some time ago. His branch is still interesting.

[1] https://github.com/pior/bottle

@sc68cal
Contributor
sc68cal commented Apr 26, 2011
    For a reason, I suppose. It is not that easy to get compression right. You need to:
    ...
    And many other edge cases I don't remember. I currently do not now a WSGI middleware that does 
    all this in a satisfying way. 
    It is usually better solved by a web-server module (lighttpd+mod_compress or Apache+mod_deflate).

Agreed. I think re-inventing the wheel to solve all these edge cases is a waste of resources, it would be much better to document the advantages and disadvantages to WSGI middleware, and then also mention web-server modules that implement it better, while linking some middleware projects in the docs. I'd be happy to implement that.

@defnull
Member
defnull commented Apr 26, 2011

I am awaiting your pull request :) (and closing this issue)

@defnull defnull closed this Apr 26, 2011
@sc68cal
Contributor
sc68cal commented Apr 26, 2011

Actually, could you reopen it? That way when you merge the pull request you can put in a "closes #92" in the message and it'll link up the history.

@defnull defnull reopened this Apr 26, 2011
@sc68cal
Contributor
sc68cal commented May 12, 2011

Sorry for the noise, rebased off your master then pushed again. I'm rushing too much and getting sloppy.

@sc68cal sc68cal added a commit to sc68cal/bottle that referenced this issue May 12, 2011
@sc68cal sc68cal First crack at #92 8bc09c2
@sc68cal sc68cal added a commit to sc68cal/bottle that referenced this issue May 12, 2011
@sc68cal sc68cal Fix link for cherrypy, issue #92 ee6f615
@sc68cal
Contributor
sc68cal commented May 12, 2011

sighs - Ok - this time issue_92 is a clean branch you can merge. Had a commit that belongs in a different pull request.

@defnull defnull closed this May 12, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment