New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Windows Build Tree still slow in 0.1.3 w/ symlinking #2680
Comments
do you have an SSD or an HDD. It is true I have only tested on windows 8, maybe something is funky in windows 7. Something that would help greatly is a dummy app with many dummy files that causes the same problem for you, that we debug it easily. |
OS X version Initial build
Incremental Build:
|
i7 64-bit laptop with SSD |
@truenorth can you also share the #'s of various files. |
ya that's bonkers then... Would love to get my hands on this app to see what we can figure out. |
Here's what our /app folder looks like
|
I'll also note that my local brocfile skips a couple steps that run on the Mac machines...I have minifyJS and wrapInEval set to false (because I can't debug the app locally with those set to true), and we have a macro-substituted tree that I don't build (because it doesn't work in windows). So in reality, my build would take even longer if I was using the exact same settings as the rest of the team. |
also, what is |
Biggest time is merging bower_components/vendor. What is the size (by count and filesize). Do you have any significant extraneous files in there?= |
man i wish broccolijs/broccoli#219 was in release broccoli, we could pretty easily see what is being slow... |
@stefanpenner broccoli-macro performs substitutions var macroSub = require('broccoli-macro');
var concatenated = macroSub(sourceTree, {
macros: [{
name: '<!-- @something-random -->',
value: 'the value which will be substituted in'
}],
glob: ['**/*.js','**/*.html', '**/*.json'],
verbose: true
}); We use this to have some per-deployment configuration variables (i.e., which API endpoint to hit). @rwjblue, here's our vendor folder
I'll update my post with file sizes |
@truenorth is the source public, i would like to review it. |
@truenorth so just confirming you guys actually have 2370 JS files. |
That sounds about right, @stefanpenner |
@elwayman02 - Can you remove unused files from bower_components? For example, only leave files that are explicitly added via |
@rwjblue that's going to take a while to sort through...we have about 45 bower dependencies 🎉 Will try to check it out, though. |
@elwayman02 - Sorry to be a pain. Our merging algorithm is fairly good (we symlink as high up the chain as possible), but unused files have in the past exacerbated many perf issues. It is also possible that somewhere in the dep tree you are not getting the new Windows symlink aware |
Node v0.10.28 |
unrelated, but let my strongly advice you upgrade to the latest 2x of npm. Many bug fixes. |
@rwjblue for reference asking for ± % npm version !10055
{ http_parser: '2.3',
node: '0.11.14',
v8: '3.26.33',
uv: '1.0.0',
zlib: '1.2.3',
modules: '14',
openssl: '1.0.1i',
npm: '2.1.7',
'ember-cli': '0.1.4' } |
honestly, just remove them entirely and see how it affects the build. Even if this breaks your app it will hint to the magnitude of perf slowness vendor or bower_components is introducing. |
@stefanpenner - I believe it to be a potential factor here. NPM 1.x does not promote deps to the latest version the same way as NPM 2.x. This means that the incorrect (aka old) version of |
good point. We should likely consider explicitly exploding if people try to start the app with an old NPM |
@elwayman02 running |
I have npm version 1.4.28 on a cli 0.1.2 app. If I clear my node_modules and run
|
Yes, you will also need to |
Deleted npm/bower folders, re-installed dependencies. I'm on symlink-or-copy 1.0.1 for sure this time.
Tried updating npm but it just gave me the same version again, lol. Going to try deleting un-imported files as suggested to see what happens. |
it may be the shear number of roots we are importing from your vendor tree. If we can omit those, and see the result it will help give us more info. |
@rwjblue this isn't looking feasible...build keeps failing because of stuff I deleted that wasn't directly imported via app.import. Broccoli-sane-watcher is going insane. |
explain this further please. How is that reproduction example coming? |
@stefanpenner I just meant that it's throwing a lot of errors about trying to watch files that don't exist (because I was trying to delete everything that isn't explicitly imported in the brocfile). I haven't been able to create a version of our app with files deleted where the build process actually completes yet. |
@elwayman02 hows the example project coming? |
I haven't had time to work on it...will try something this week hopefully. I'm running out of time before they take my windows machine away, so I need to do it soon. =P |
@stefanpenner in the meantime I ran ember server on our sample project that we've put together to help teams get started with an ember app quickly. It has about 28 files and maybe 2 dozen dependencies; a lot lighter than our actual project.
The build time was still incredibly long for how lightweight the application is. Maybe we can work from this sample application rather than my main one, which is a lot harder to pick apart? |
Package.json (edited to remove 3 internal dependencies):
Bower.json (removed 1 internal dependency)
The main thing missing is that we have an internal version of bootstrap included as well, which adds some heft to the dependency chain. Brocfile.js
|
ya still seems long, my windows clients app is hundreds of modules and many dependencies and they have 2s incremental builds. Wish i could just have your machine infront of me. |
@stefanpenner you're gonna love this...we recently upgraded our main app to 0.1.5 with ember-cli-6to5 brought in...now my build times are around 700,000 ms...nearly tripled xD |
so after investigating @elwayman02 machine, there where some issues.
@felixrieseberg this is likely a topic we should chat about when it comes to windows ergonomics. |
Does any have a ballpark figure of how the speed compares with a similar spec mac and pc with 8.1 with ssd. Lets say the mac is the new pro 13 2015. |
I just updated my ember-cli in my project from 1.13.1 to 1.13.8, and the buildtimes went up considerably, I used to get build times on the same project with literally nothing changed (except for the updates of ember-cli) of about 2 seconds, but after the update, I'm looking at 2 minutes (!) build time. Tested on Windows 10 - 6GB Ram - SSD. To get an indication what changed,
Changes in dependencies in package.json
|
@pjcarly it's probably best to create a new issue for this |
2 minutes === symlinks aren't setup, please be sure to review the windows guide on the website. I was testing windows 1.13.0 and 1.13.8 and master all weekend and was not seeing this regression (even on large apps). |
Awesome, it was symlinks indeed. For anyone finding this comment in the future, the fix for windows is setting up Symlinks: |
@pjcarly master of cli actually tests and logs this. I suspect this will help future issues. |
Opening issue at the request of @stefanpenner.
I recently following the instructions to allow symlinking in Windows (7 Pro) found here: http://www.ember-cli.com/#symlinks-on-windows. After updating my build, the build time went from around 230,000 ms to about 140,000 ms. Still a lot higher than expected, so posting this issue to see what we can do to track down a problem.
Snapshot of some of the relevant dependencies:
I've included the build output of a
Initial Build:
Change Build 1:
Change Build 2:
The text was updated successfully, but these errors were encountered: