Can we improve upon the sass compiler performance? #313

Open
AntelopeSalad opened this Issue Jan 30, 2014 · 19 comments

Comments

Projects
None yet
6 participants
@AntelopeSalad

First off, thanks a lot for django-pipeline because I cannot imagine developing with django without it.

Here's the situation:
Today I decided to switch a project to using sass and in development mode my requests went from taking ~30-40ms to 1.5 seconds. It's also no longer possible to use live reload tricks which let me adjust the css without reloading the page.

You might think 1.5 seconds is not a lot but this is a trivial setup at the moment with nothing but bootstrap 3.1 being pulled in but I don't think it will gain much more time. Even with a nearly empty scss file and no bootstrap it takes 1.3 seconds to load a page.

Just simply clicking around my app going page to page feels really slow and it's slowly killing me inside.

With debug false and after running collectstatic then things are fast like usual because the sass files have been pre-compiled to css (right?), so there's no problem here.

With rails on the same machine using sass and coffee script the same requests happen in 100ms or so in development mode and it's handling the scss/coffeescript conversion to css/js behind the scenes in between page views. It feels very seamless to develop against.

Are there any strategies we can implement to improve the sass compiling time in development mode? Technically it appears possible because it's not delayed in rails at all.

@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Jan 30, 2014

Member

We need to change how pipeline compile files, both for development and production, something like #299, but with better use of cache, and how compilers works, maybe inspired by django-staticbuilder.

Sadly, I have very little time to do so 😑

Member

cyberdelia commented Jan 30, 2014

We need to change how pipeline compile files, both for development and production, something like #299, but with better use of cache, and how compilers works, maybe inspired by django-staticbuilder.

Sadly, I have very little time to do so 😑

@AntelopeSalad

This comment has been minimized.

Show comment
Hide comment
@AntelopeSalad

AntelopeSalad Jan 31, 2014

Are you saying pipeline is adding overhead in production per request, or are you only talking about the time it takes to run collectstatic?

Are you saying pipeline is adding overhead in production per request, or are you only talking about the time it takes to run collectstatic?

@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Jan 31, 2014

Member

No, in production pipeline is just using staticfiles to point to file in STATIC_ROOT, there is no overhead.
But for both development and collectstatic, we don't really make a good job at playing well with compilers.

Member

cyberdelia commented Jan 31, 2014

No, in production pipeline is just using staticfiles to point to file in STATIC_ROOT, there is no overhead.
But for both development and collectstatic, we don't really make a good job at playing well with compilers.

@AntelopeSalad

This comment has been minimized.

Show comment
Hide comment
@AntelopeSalad

AntelopeSalad Jan 31, 2014

Is the general improved strategy for development to somehow store the mtime of each sass file and only rebuild the css per request if the css has changed?

Is the general improved strategy for development to somehow store the mtime of each sass file and only rebuild the css per request if the css has changed?

@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Jan 31, 2014

Member

In the case of sass and others, it's more complicated, because of @import.

Member

cyberdelia commented Jan 31, 2014

In the case of sass and others, it's more complicated, because of @import.

@AntelopeSalad

This comment has been minimized.

Show comment
Hide comment
@AntelopeSalad

AntelopeSalad Jan 31, 2014

I'll take a look at rail's source tomorrow and see how they are doing it. Hopefully it's something I/we/community can apply to pipeline.

I'll take a look at rail's source tomorrow and see how they are doing it. Hopefully it's something I/we/community can apply to pipeline.

@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Jan 31, 2014

Member

@AntelopeSalad Outside of rails, look at django-staticbuild and issue #299 (and the PR that goes with it).

Member

cyberdelia commented Jan 31, 2014

@AntelopeSalad Outside of rails, look at django-staticbuild and issue #299 (and the PR that goes with it).

@AntelopeSalad

This comment has been minimized.

Show comment
Hide comment
@AntelopeSalad

AntelopeSalad Jan 31, 2014

I don't really know what to make from 299's PR, it's only a 1 line change?

The staticbuild's middleware uses a time comparison, but you said that won't work because of imports.

I don't really know what to make from 299's PR, it's only a 1 line change?

The staticbuild's middleware uses a time comparison, but you said that won't work because of imports.

@charredUtensil

This comment has been minimized.

Show comment
Hide comment
@charredUtensil

charredUtensil Feb 12, 2014

+1 and ideas:

  1. Is it possible to add a setting to django-pipeline for a directory to watch and, if all files in the directory are unchanged, don't recompile? I have all my scss files in my project's assets directory. Sure, editing one file would cause them all to be recompiled, but unless you're doing heavy styling work, this should solve the problem in all other cases. In any case, this would need to be done before starting Sass, as it seems loading Sass + all of its dependencies seems to be the key limiting factor
  2. Is it possible to keep Sass running as a daemon with the --watch option while the development server is running? That way, Sass would take care of itself automatically in this regard.
  3. Can we abuse the .sass-cache directories to get a list of files to "track"?

+1 and ideas:

  1. Is it possible to add a setting to django-pipeline for a directory to watch and, if all files in the directory are unchanged, don't recompile? I have all my scss files in my project's assets directory. Sure, editing one file would cause them all to be recompiled, but unless you're doing heavy styling work, this should solve the problem in all other cases. In any case, this would need to be done before starting Sass, as it seems loading Sass + all of its dependencies seems to be the key limiting factor
  2. Is it possible to keep Sass running as a daemon with the --watch option while the development server is running? That way, Sass would take care of itself automatically in this regard.
  3. Can we abuse the .sass-cache directories to get a list of files to "track"?
@charredUtensil

This comment has been minimized.

Show comment
Hide comment
@charredUtensil

charredUtensil Feb 12, 2014

Another idea:

  1. If this is so much faster in Rails, how about a Rails app that runs alongside the Django one in development mode? Just proxy requests for stylesheet files over to it.

Another idea:

  1. If this is so much faster in Rails, how about a Rails app that runs alongside the Django one in development mode? Just proxy requests for stylesheet files over to it.
@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Feb 19, 2014

Member

@AntelopeSalad They are incomplete but interesting solutions, #299 would make sass like compiler work almost like intended, but wouldn't work for development. django-staticbuild try do don't put that much intelligence in the process, leaving the compiler trying to be smarter.

Member

cyberdelia commented Feb 19, 2014

@AntelopeSalad They are incomplete but interesting solutions, #299 would make sass like compiler work almost like intended, but wouldn't work for development. django-staticbuild try do don't put that much intelligence in the process, leaving the compiler trying to be smarter.

@cyberdelia cyberdelia added this to the 1.4 milestone Mar 20, 2014

@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Mar 22, 2014

Member

@AntelopeSalad @charredUtensil Could you try your setup against #326?

P.S: The templatetags names have change.

Member

cyberdelia commented Mar 22, 2014

@AntelopeSalad @charredUtensil Could you try your setup against #326?

P.S: The templatetags names have change.

@charredUtensil

This comment has been minimized.

Show comment
Hide comment
@charredUtensil

charredUtensil Mar 23, 2014

@cyberdelia Tried running that. I see no performance benefit whatsoever from the 1.4.x branch. It's still forking tons of sass processes, four at a time, for each .scss file on every page load. On my main development box with RVM and lots of other Ruby stuff installed, sass takes a little longer to start each time. I also have a VM with a more plain Ruby installation that can start sass quicker, but this is less than ideal. I just dropped some other old Python packages with bizarre, obsolete dependencies, and am trying to get rid of the VM in favor of developing this app locally again, but I'm still seeing 10-20s page loads with django-pipeline.

@cyberdelia Tried running that. I see no performance benefit whatsoever from the 1.4.x branch. It's still forking tons of sass processes, four at a time, for each .scss file on every page load. On my main development box with RVM and lots of other Ruby stuff installed, sass takes a little longer to start each time. I also have a VM with a more plain Ruby installation that can start sass quicker, but this is less than ideal. I just dropped some other old Python packages with bizarre, obsolete dependencies, and am trying to get rid of the VM in favor of developing this app locally again, but I'm still seeing 10-20s page loads with django-pipeline.

@cyberdelia

This comment has been minimized.

Show comment
Hide comment
@cyberdelia

cyberdelia Mar 24, 2014

Member

@charredUtensil With this new version, @import should world, so you should be able to point at only at couple of files, avoiding starting too many multiple processes.

Member

cyberdelia commented Mar 24, 2014

@charredUtensil With this new version, @import should world, so you should be able to point at only at couple of files, avoiding starting too many multiple processes.

@charredUtensil

This comment has been minimized.

Show comment
Hide comment
@charredUtensil

charredUtensil Mar 24, 2014

@cyberdelia Better, but it's still calling sass once on every request, even though I haven't modified any files.

@cyberdelia Better, but it's still calling sass once on every request, even though I haven't modified any files.

@danxshap

This comment has been minimized.

Show comment
Hide comment
@danxshap

danxshap Jul 5, 2014

Joining the conversation pretty late here but wanted to check in and see what you all have been doing for this.

@AntelopeSalad have you come up with your own solution or are you still dealing with super long requests in your dev environment?

I like @charredUtensil's idea number 2 regarding taking advantage of Sass's --watch option. Ideally: you save a change to a scss file, Sass automatically recompiles the CSS, and by the time you refresh your browser it's already sitting there ready to be loaded, right?

@cyberdelia Couldn't this be as simple as an optional setting that tells SASSCompiler not to run compile_file() when DEBUG is False? Only when this setting is enabled, it can optimistically assume the developer/user is running their own sass --watch and thus that the compilation is already done.

Thoughts?

danxshap commented Jul 5, 2014

Joining the conversation pretty late here but wanted to check in and see what you all have been doing for this.

@AntelopeSalad have you come up with your own solution or are you still dealing with super long requests in your dev environment?

I like @charredUtensil's idea number 2 regarding taking advantage of Sass's --watch option. Ideally: you save a change to a scss file, Sass automatically recompiles the CSS, and by the time you refresh your browser it's already sitting there ready to be loaded, right?

@cyberdelia Couldn't this be as simple as an optional setting that tells SASSCompiler not to run compile_file() when DEBUG is False? Only when this setting is enabled, it can optimistically assume the developer/user is running their own sass --watch and thus that the compilation is already done.

Thoughts?

@danxshap

This comment has been minimized.

Show comment
Hide comment
@danxshap

danxshap Jul 6, 2014

@cyberdelia curious to hear your thoughts on this 6c95ac3

Pretty simple/harmless, and it's allowing me to use sass with pipeline in my dev environment with 10x faster requests. In my dev settings only, I add PIPELINE_DISABLED_COMPILERS = ('pipeline.compilers.sass.SASSCompiler',). Pipeline still renders the output file source to my page, but it never actually runs the Sass compiler and assumes the file has already been compiled.

I have sass --watch running in another terminal so that whenever I change/save a .scss file it's automatically compiled.

And if I'm not working with CSS at the moment, I don't even have to worry about running that command. After all, why do I need pipeline to recompile my Sass files on every request if I'm not working on CSS?

Unless you know of a better solution, I will definitely be using this going forward!

danxshap commented Jul 6, 2014

@cyberdelia curious to hear your thoughts on this 6c95ac3

Pretty simple/harmless, and it's allowing me to use sass with pipeline in my dev environment with 10x faster requests. In my dev settings only, I add PIPELINE_DISABLED_COMPILERS = ('pipeline.compilers.sass.SASSCompiler',). Pipeline still renders the output file source to my page, but it never actually runs the Sass compiler and assumes the file has already been compiled.

I have sass --watch running in another terminal so that whenever I change/save a .scss file it's automatically compiled.

And if I'm not working with CSS at the moment, I don't even have to worry about running that command. After all, why do I need pipeline to recompile my Sass files on every request if I'm not working on CSS?

Unless you know of a better solution, I will definitely be using this going forward!

@kviktor

This comment has been minimized.

Show comment
Hide comment
@kviktor

kviktor Feb 27, 2015

Any progress on this? I've been using the "patch" of @danxshap and it's been working great.

kviktor commented Feb 27, 2015

Any progress on this? I've been using the "patch" of @danxshap and it's been working great.

berkerpeksag added a commit to berkerpeksag/pythondotorg that referenced this issue Mar 15, 2015

Use DummySASSCompiler in DEBUG mode to fix performance issues.
Currently, django-pipeline runs sass in every requests. Every
request took ~10 on my computer. Until jazzband/django-pipeline#313
gets solved, use a dummy compiler class and run "sass --watch"
in a separate terminal.
@svengt

This comment has been minimized.

Show comment
Hide comment
@svengt

svengt Apr 7, 2015

I had the same problem till a colleague told me about sassc and libsass. This sass compiler is made in C which makes it ten times faster. I followed these instructions to install it on my Mac (brew has some older versions) https://www.snip2code.com/Snippet/189303/Install-SassC---LibSass-for-Mac-OS-X-10- .

# Install SassC Interpreter
$ cd /usr/local/src
$ curl -kL https://github.com/hcatlin/libsass/archive/master.zip > libsass.zip
$ unzip libsass.zip

# Install LibSass Library
$ cd /usr/local/src
$ curl -kL https://github.com/sass/sassc/archive/master.zip > sassc.zip
$ unzip sassc.zip

# Compile SassC
$ export SASS_LIBSASS_PATH="/usr/local/src/libsass-master"

$ cd sassc-master
$ make
$ sudo ln -s $(pwd)/bin/sassc /usr/local/bin/sassc
$ chmod +x /usr/local/bin/sassc

# Example Usage
$ /usr/local/bin/sassc ~/Desktop/input.scss ~/Desktop/output.css

Next, you just add these settings and it works a lot faster

PIPELINE_COMPILERS = (
    'pipeline.compilers.sass.SASSCompiler',
)

PIPELINE_SASS_BINARY = '/usr/bin/env sassc'

svengt commented Apr 7, 2015

I had the same problem till a colleague told me about sassc and libsass. This sass compiler is made in C which makes it ten times faster. I followed these instructions to install it on my Mac (brew has some older versions) https://www.snip2code.com/Snippet/189303/Install-SassC---LibSass-for-Mac-OS-X-10- .

# Install SassC Interpreter
$ cd /usr/local/src
$ curl -kL https://github.com/hcatlin/libsass/archive/master.zip > libsass.zip
$ unzip libsass.zip

# Install LibSass Library
$ cd /usr/local/src
$ curl -kL https://github.com/sass/sassc/archive/master.zip > sassc.zip
$ unzip sassc.zip

# Compile SassC
$ export SASS_LIBSASS_PATH="/usr/local/src/libsass-master"

$ cd sassc-master
$ make
$ sudo ln -s $(pwd)/bin/sassc /usr/local/bin/sassc
$ chmod +x /usr/local/bin/sassc

# Example Usage
$ /usr/local/bin/sassc ~/Desktop/input.scss ~/Desktop/output.css

Next, you just add these settings and it works a lot faster

PIPELINE_COMPILERS = (
    'pipeline.compilers.sass.SASSCompiler',
)

PIPELINE_SASS_BINARY = '/usr/bin/env sassc'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment