need intermediate files when gzip'ing

[mixmastamyk]

Hi,

I have similar to the following:

env.register('css', 'css/layout.css', 'css/day.css', 
    filters=['cssprefixer', 'cssmin', 'gzip'], 
    output='css/wa/site_min.css.gz')

The problem is that only the gzip file shows up. I need the css file as well to be in the folder so the webserver can find it (for browsers that don't accept gzip). I'll try making two rules, but seems like it should be an option somehow.

[miracle2k]

Indeed I think you currently need to define two rules. You may able to use nested bundles:

 uncompressed = env.register(...., filters=['cssprefixer', 'css'])
 compressed = env.register(uncompressed, filters=['gzip'])

If you enable the cache, then the second bundle will have little additional performance cost.

As for allowing an option to create two files on the fly - the key here is "somehow". I can see it being added, if we find a smart, well-designed way to do so.

[mixmastamyk]

Hmm,

If it helps, the way I was expecting it to work is similar to the make-style system I've used prior to experimenting with webassets. So as the data is passing thru it would create an intermediate file for each operation down the chain. Each of the intermediate files might have a default filename using the base of the bundle name (I guess), and an extension that made sense for the operation (cssmin --> _min.css). Keyword args might modify if needed.

I did find a . folder which is likely the cache you speak of. However as it contains hashed filenames I can't tell which file is which, and when do they get deleted? Seems a like a bit of black magic. In comparison, the make-style system is clear in this respect. I can see the files and timestamps, delete a few, and they are available to serve if need be.

The drawback to it of course, being lots of files making a mess in the folder. The way I sidestep that is to put all the output files into their own subfolder. If there were a keyword arg for output folder and it was highlighted in the docs I think it could work. There would be the clarity of make operation, with the elegance of using python instead of (clear as mud) makefiles.

[miracle2k]

I'm not convinced that having an output file for each step of processing lying around is particularly useful. Your gzip scenario is a special case (In fact, I've always considered gzip an example filter in a way, given that most people will let the webserver do the compression).

(By the way, I'd be interested in your webserver config with regards to serving the gzipped output of webassets.)

Consider also that webassets works in two steps - it first applies "input filters" to all the source files, then concats everything, then applies output filters to the whole thing. Should each input filter step also create an intermediate file file? If yes, you're now going to have a lot of them. If not, this contradictory behaviour might be confusing.

The cache currently is never cleared. That's an area that needs looking at.

Anyway, two thoughs here:

First, at some point I'd like to add the ability to override environment configuration values on a per-bundle basis. If the cache system could be written to use clear, readable filenames, this would basically allow you to do what you're suggesting.

Second, given this configuration:

Bundle(
     Bundle(
         ....,
         filters=['cssprefixer', 'css'],
         output="uncompressed.css"
     ),
     filters=['gzip'],
     output='compressed.css'
)

The "output" argument of the nested bundle, "uncompressed.css", is currently ignored. Maybe an elegant solution would be to change that behavior, and have both files created. In fact, the more I think about this, the more I like it.

[mixmastamyk]

I agree when everything is working it isn't very valuable. But when something is broken, being able to look at each step helps to pinpoint bugs.

Here is the setting I use with the nginx web server. One of the reasons I precompress is to provide acceptable performance on a very resource constrained virtual server.

gzip_static on;
http://wiki.nginx.org/NginxHttpGzipStaticModule

Before serving a file from disk to a gzip-enabled client, this module will look for a precompressed file in the same location that ends in ".gz". The purpose is to avoid compressing the same file each time it is requested.

Your solution appears to be a simple way to get from point A to B without disruptive changes.

(p.s. the second output file should end in .gz, no?)

[miracle2k]

As a reminder to myself (or anyone who wants to implement this): Another future feature intersects here: Including a bundle's compressed result directly inside a template. Should a nested bundle still be written to a file if the parent bundle is "built to memory", so to speak?

[miracle2k]

One more thing to consider: A nested bundle with an output target and bundle.debug=False is what the docs currently suggest when using CSS compilers. People using this setup would now receive two output files in production where before they had only one. This may or may not be a problem.