-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incredibly slow [initial] build times with ember-cli 0.2.1 #3637
Comments
You need to consider 2 different build-phases when considering performance. This will help me understand if there is a regression or not.
It seems your large issues are on initial build, that is currently expected since we do a clean build on re-build, to avoid invalid builds. There are plans to introduce a cache, but their exists complexity in knowing when to blow away that cache. Two slow parts of initial build are:
If you do not want to use babel for your app, it is not currently required you can safely remove it from your package.json Also, it appears you are on windows please be sure to thoroughly review: http://www.ember-cli.com/#windows we have a recent submission from the lovelly Microsoft OSS engineer @felixrieseberg which aims to try and fix-up some windows issues. |
the positive thing is, if we introduce a cache for babel (although it is tricky to know when to evict that cache) initial builds will drop by nearly 30s. I know my co-workers would also really appreciate that, as one of our biggest apps is about 70kloc. |
@thomasjmwb: Thanks for reporting. I'd love to see if EDIT: I just saw that you already used ember-cli-windows. This makes things a bit more curious... I'll see what kind of build times I see. |
@felixrieseberg i believe incremental are within tolerable levels, I suspect he recently got babel as a dependency and ways surprised by the upfront cost of a non-cached babel build. |
I believe the only resolution to this, is enabling a persistent cache for the big offenders, sourceMapConcat and babel. We will have to nail down the heuristics we feel are acceptable for such a cache and implement. |
Here are my numbers:
I'm running preview bits of Windows 10 and have full encryption activated, so those numbers should be a bit smaller on a non-encrypted drive with an optimized build. @thomasjmwb: Do me a favor and install the latest version of @stefanpenner: I think the numbers (on my system) are totally fine. Incremental builds of a second are totally within range (and basically identical with my MBP13's build times). |
so this seems like the root of your/the problem |
it is certainly an issue, but we are seeing similar rebuild times across other environments that do not run into the ENOTEMPTY error. My team member on a OS X is getting rebuild times of 4~6 seconds (awesome!) while my other team member on ubunto 14.04 is getting rebuild times of 19 seconds. |
@thomasjmwb are you on an SSD or HDD? |
those number still seem pretty high. |
We have a pretty large app (2k files, 125k sloc) On a 2 core i7 macbook (8gb ram - ssd) initial build 105000 ms (details below) While the initial build time is painful, the rebuild times hovering around 10 sec is reasonable. We did disable hinting I've spent a lot of time debugging broccoli trees and looking at addons invalidating trees w/o needing to. Current pet project: es6 opt-in so power of babel but only as desired/required Initial build:
Change one .js file:
After removing babel: (clean rebuild)
single js file change w/o babel:
|
HDD. my tmp folder is now at ...5.87Gb. I'm gonna go ahead and restart my computer and see what the build time is then. Possible unrelated detail i'd like to volunteer in case it happens to be relevant.. we have a couple files that are ~3MB each (very large JSON trees in fixture data). Perhaps these files contribute to such a huge tmp directory? |
The huge temp dir is the result of a crashing process or control-c and we may not be doing the correct cleanup? HDD will likely get a bit faster with some planned work, but that is likely the source of the somewhat slowish builds. We have some WIP that will reduce the amount of internal work we do dramatically. I suspect it will help the HdD users more then the SSD users. |
Could you elaborate a bit more on that? Or just point me to whatever resources you checked out while debugging broccoli trees? I want to start seeing if I can optimize our build process but I'm unsure of where to start since it's hard to tell just from the ember-cli docs what I'm getting out of the box. For example, how I should go about changing how often we recompile test/app/vendor files. |
I don’t have a list of resources, rather hard won insights. running For instance, I was having slower rebuild times and determined that an addon was not caching its’ result and therefore on every rebuild was forcing a significant cache invalidation. Adding caching in my addon resolved that. Looking at the sane watcher/slow trees output alone can help focus. We had an issue where saving a .js file was causing the SassCompiler to run - reasoning about that led to an addon which turned into an ember-cli patch for postprocessTree Finally use of broccoli-stew https://github.com/stefanpenner/broccoli-stew https://github.com/stefanpenner/broccoli-stew - specifically log (and the new HTH
|
Thanks a ton @jschilli ! I'm also checking out https://github.com/ember-cli/ember-cli/blob/master/lib/broccoli/ember-app.js as a good entry point to get an idea of what happens after dropping a bunch of stuff in my app's Brocfile. |
yeah, reading the source and understanding the flow helps a lot in debugging perf issues
|
Update-- After making some pretty drastic changes to try and get some reasonable windows performance this is what our brocfile looks like:
Pretty drastic differences, all running the exact same code. The resident windows guru on our team seems to think the primary cause of the slow windows performance is the heavy I/O occurring during rebuild. I'm not really sure what other information to provide, please let me know if theres anything else that someone might find useful. |
Some additional stats on windows ember serve with DEBUG="*":
|
@thomasjmwb impressive gains. heavy i/o and HDD is almost certainly contributing to slow(er) build times on windows. One thing that I noticed is I have on my list to look at the @thomasjmwb are you using es6 features? have you tried running without babel? |
We are only using es6 module syntax at this time. We considered ditching babel, but our estimated gains (based on the Slowest Trees table logged out at the end of a build) did not make it seem like it was worth the effort considering we have ~1000 files to futz with. @ tests, agreed, it is definitely not ideal. we are planning on doing something where you add '-e tests' to ember serve to enable tests so developers can go that route at their discretion |
@bpcrao it looks like you may not have followed the instructions @ http://www.ember-cli.com/#windows although maybe you have and are just being trolled by a combination of not having an HDD, being on windows, a bug introduced when source maps where added (causing more things to be rebuilt then expected). |
addons. |
tried with SSD, build and rebuild improved by 50% build 25 sec |
this is expected, although some work exists (i still need to port it to windows before i can release it) to make warm boots much faster. for reference: https://github.com/stefanpenner/broccoli-persistent-filter
This is still somewhat expected, we regressed when adding source maps. We are doing work now to once-again reduce this. I also believe the work to fix the regression we introduced with source maps, will nicely improve the HDD case. |
are you able to create symlinks ? can you check with npm package can-symlink . |
@bpcrao yes I am able to build symlinks I figured out what was going on, because of emberjs/ember.js#10310 we are building our own patched version of ember in the bower_components directory. The result being that the bower_components directory is a lot bigger then it should be because of the npm and bower packages that get installed. After cleaning this up the initial build time is now 46787ms Still slow imo but much better then before. |
@tdesmet |
According to this.
Can anyone confirm this or point to where I can check this myself in the source code? |
This isn't entirely true, Broccoli is like react.js where it "rebuilds" the world, but unless bugs exists (some still do) only the changes need to be processed. For windows users, our friends at Microsoft (who use ember and ember-cli) have provided an add-on, to correctly adjust windows defaults to allow for symlinks see: http://www.ember-cli.com/#windows. This ultimately enables same-order performance times to comparable posix systems. One thing worth noting, recently we had a performance regression in our concat phase. I fixed part of it last weekend, and plan to fix the next chunk this weekend. Hopefully soon that regression will be gone, For reference, one of our apps at work has > 120,000 loc of just app code, and a huge number of addons. Without my fixes:
this is extremely poor With my fixes:
The fixes are a bit tricky, so the work require to push the changes through the whole ecosystem has taken more time the expected. Luckily one of the blockers, the stable broccoli-plugin api landed (last week). Which will help us get the ball moving again :) Sorry, for the current regression of performance, we will sort it out again shortly. |
It should rebuild only what's dirty, which is what I believe React does. Isn't this what Broccoli does too?
How far back would one need to downgrade ember-cli in order to note any perf improvements? Even if not a real solution, it would help our team get some relief. |
Do you mind sharing some details on this? Like some of the places where you think bottlenecks could occur or just any general hints as to where start looking? Thanks. |
I'll have to verify my recent tunings have improvement windows performance (i'll likely do so tomorrow). TL;DR turbo button enabled for many previously pathological cases. given one specifically poor example, on my laptop (OSX SSD) original times:
with fixes times:
and 1,600 of the remaining time looks like more low hanging fruit. |
Just tried ember-cli from latest master on windows, it has similiar build time as 1.13.7 for the below testing repostory: Initial build:
Incremental build:
There is 1580 directories in the tmp directories , while there is only around 8XX directories in 1.13.7 |
@tsing80 it looks like your app doesn't hit any of perf issues the recent PR's have been targetting. (large bower_components/vendor trees); That being said, it may have not included some of my latest changes to Funnel. (or maybe its slow IO on windows – which i have some ideas for) It is hitting 2 major areas I hope to land solutions for shortly:
Some quick questions:
example app running on OSX
|
of note, the original posting issue will likely be dramatically improved by persistent caches in broccoli-filter. for reference: https://github.com/stefanpenner/broccoli-persistent-filter it is still pending windows support before it can be released. Some numbers, at work where have been using persistent-filter for babel, which reduces our initial build babel times from 60s -> 2s, a very welcome speedup. |
@bpcrao I believe you issue will be resolve (or mitigated) as a result of #3637 (comment) |
@tsing80 I have also noticed something, ember-cli-jquery-ui is looping over files in the directories in-order to import them. This is resulting in an unexpectedly high number of A quick glance suggests if I where to add globbing support to This seems like a great little area to improve I have opened: #4644 to specifically track what i see, and describe a set of proposed solutions |
|
for those interested, come help populate the stress app ember-cli/stress-app#2 |
upgrading the following libs will help alot:
This will also come out with the next CLI release by default. TL;DR your initial build will be slightly slower, but subsequent restarts will have a primed persistent cache to feed off. To prevent the cache from growing onbounded:
To prevent getting trolled by stale cache:
Is this code super new and scary? Not really, we have been using it internally for the last month or two without trouble. |
@stefanpenner I'd like to reopen this issue. We posted a question here: http://discuss.emberjs.com/t/ember-cli-compilation-time/8834 We have spent quite a bit of time updating all our dependencies to the latest versions in hopes compile time would be faster. We are using 1.13.8. Our initial load is ~144000ms
And subsequent changes are over 20s.
Our deps:
We cannot find what is taking so much time. We do have a large app (~1000 files) but it's really showing that ember-cli is having a hard time keeping up with large apps like ours making development very difficult and time consuming. Thoughts? |
We are working on it. Making some good progress the next release will be noticeably better. Feel free to follow along on the stress test repo. And it will continue on those lines. Ultimately if you have some extremely costly filters that don't have a persistent cache. Boot is going to be slow. Please be sure to open issues on offending broccoli-plugin issue trackers |
I'm crying over this, I wanted to give Ember Js another try but the slowness of build is somehow pushing me back for our new project. I can't do anything productive with ember right now, because every change I do make my workspace freeze literally. |
@stefanpenner when is 13.9 coming out? |
Hmmm, emeber 2.4.2 with the latest stuff, in docker... But compilation time, oh man
|
@rokka-n if you have perf issue. Please read the perf_guide.md in this repo |
Thank you. I looked through it, I guess its mainly saying about npm packages being up to date. I've created container to run ember, npm and some other tools. https://github.com/rokka-n/containers/tree/master/docker_ember_cli I'm curious if anybody can get substantially better results for ember server, than mentioned above. |
after creating a new ember app I'm getting what I think is very slow build time.. with a new app I see the following:
rebuild:
What should I expect build time for a pretty small app to be? I'm seeing this problem magnified in a much larger app:
Some more context around the big app:
Our Brocfile.js / bower.json:
https://gist.github.com/thomasjmwb/b4bbdde5e9c1a972de7d
file # stats:
total files: 1045
./pod-a 41
./pod-b: 52
./adapters: 29
./pod-c 52
./pod-d
./pod-e 46
./pod-f 12
./pod-g 184
./pod-h 16
./commons: 16
./components: 50
./controllers: 18
./data-access: 2
./pod-i 14
./helpers: 1
./mixins: 27
./models: 15
./pod-k: 64
./properties: 19
./routes: 14
./serializers: 31
./pod-l: 48
./styles: 67
./pod-m: 21
./templates: 94
./transforms: 14
./views: 59
I am on windows, but our developers on mac and linux are both seeing long build times like i've described. I've also tried the prescribed solutions for windows, using ember-cli-windows-addon, making sure i dont have virus scanners running etc.
Could someone point out some potential pitfalls that I'm hitting with a large project?
The text was updated successfully, but these errors were encountered: