Performance Fixes: from the very small to very large #117

merged 6 commits into from Oct 31, 2012


None yet
4 participants

See commit messages.

twinturbo added some commits Oct 30, 2012

Reintroduce last_manifest/manifest
I had previously removed the 2 manifest concept from the code. I thought
this was ok because the tests were passing. There weren't enough tests
to cover all use cases. Two manifests are needed for tracking changes in
dynamic prerequisites between invocations. Here's the use case: you have
a file dynamic file A that imports B. Inovking A will _always_ update
B's mtime in the manifest. This causes dependency modification check
results to be false positives.
Use integer time instead of Time objects
Parsing times is slow. This takes up a lot of time when loading large
manifests. Using integer timestamps should also work better across
multiple platforms. We also completley avoid timezone math.
Completely skip computation if nothing has changed
This commit introduces a global dirty state. This check is used when
invoking a project. It checks to see if any of the known files have
changed, or if any files have been deleted or added. Projects are dirty
if the assetfile has changed. When projects are clean invoking them does
nothing. This check saves costly build time. The build tree is
constructed internally every time. This can be costly. Running those
rake tasks would not change anything anyway so that process can be by

This commit should make the preview server much faster. Say you have a
set set of assets: images, application.js, application.css, and
index.html. Opening the preview server will send an HTTP request for
index.html. This request will compile the pipeline. Further requests to
application.js, application.css and images should return almost
instantly because pipeline does not have to do any processing.
Middleware should take a project
Refactor the server/middleware to take a project instance. This change
does not instantiate a new project for every HTTP request. This prevents
needless resetting and rebuilding between requests. Mainly this ensures
that manifest objects maintain state between HTTP requests.

This build passes on Ruby 1.9.2, 1.9.3 and Ruby-Head:!/livingsocial/rake-pipeline/jobs/3001245,!/livingsocial/rake-pipeline/jobs/3001246,!/livingsocial/rake-pipeline/jobs/3001249. The other travis failures are gem install errors outside the scope of this PR. They can be fixed in another PR.

@ahawkins ahawkins referenced this pull request Oct 31, 2012


Version bump 0.8.0 #116

wycats added a commit that referenced this pull request Oct 31, 2012

Merge pull request #117 from twinturbo/performance
Performance Fixes: from the very small to very large

@wycats wycats merged commit 234f8a3 into livingsocial:master Oct 31, 2012

1 check failed

default The Travis build failed

What a "Pipline" is?


wagenet replied Nov 1, 2012

And why are you assigning an unused variable of project here?

Also, you just removed the config variable so this blows up too.


wagenet replied Nov 2, 2012

Fixed here: #122


tchak replied Nov 2, 2012

I made a PR earlier with fix for this and to railties too #121

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment