Running Meteor when AVG Antivirus is installed cause 20-30 seconds between refreshes

[running-coder]

Just wanna throw it out there...

So I've been using Meteor for quite some time now (just updated to 1.2.1) and I couldn't figure out why it was refreshing fast (~1 sec) at work and slow at home (~20-30 seconds). I just happened to have windows task manager opened while working on my app to see that AVG was taking up to 40% of my CPU upon Meteor refreshes. I silenced the AVG program and now the application loads just as fast as it should be. This is really a game changer since it was painful to refresh every time and I was quickly loosing interest for developing at home!

Perhaps this will help other people that still have the problem.

Thanks

[rozzzly]

I would imagine this is not unique to AVG. Most avs will hook JavaScript execution and will rescan every of the thousands of files that gets created when meteor compiles. One should be able to avoid this by whitelisting the meteor exe / the projects directory.

[edinabazi]

@running-coder Are you saying that you work with Meteor (at work) on Windows and you get ~1 sec refresh times? Because I've been having problems with extremely slow refresh times on Windows without having any kind of antivirus installed. And from what I've seen, everyone else has.

[running-coder]

Yes, ~1 sec refreshes at work (always been super fast on that environment) using Windows7. My issue was at home on Windows 7 also. I've tried to update to Windows 10 thinking it was maybe a firmware issue with some version I was running but even after the update I was getting the same issue. Finally found out after having the Task Manager opened by accident... The nodejs application was increasing in % (which is expected) but that AVG software was going through the roof. Perhaps that could also be a way to debug your refresh rate issue. As rozzzly mentionnend and kind of software that re-scan your files may have caused it.

[stubailo]

Does anyone have any ideas about how we can address this? I'm not sure what we could do to stop an antivirus program from slowing down Meteor.

[edinabazi]

I don't think it's an antivirus. I don't have any installed and it still happens to me. I opened task manager and noticed that the "Evented I/O for V8 JavaScript" task went up to like 15% in CPU use from 0.3% after saving a file. And this is with a freshly created project with the default files. I'm not sure if that's normal. I have an i7 CPU with 8GB of RAM and I'm writing the files on an SSD so it's definitely not the hardware's fault. I had to install a Linux distro AND make a Hackintosh just because of Meteor's poor build times on Windows.

[rozzzly]

@stubailo do you know if meteor writes temp files (if any) during compilation on Windows? Most major AVs with runtime protection hook into the kernel and intercept system calls so they monitor every file creation, process created, etc. They treat certain files differently; one facet of that is based on the directory and file type. For instance the C:\Windows\Temp folder trends to be pretty lax compared to appdata (I want to say that meteor installs its system wide binary there iirc)edit: they are stored there, which is watched a little more cautiously because malware will often hide in there. @running-coder Are you running node elevated as an administrator or as normal user? In addition, I'd love to know where you're storing your project files? In addition, your user folder (ie: C:\Users\rozzzly) should be less restrictive than say C:\proj. Another idea might be writing temp files to a .tmp instead of .js? It's possible that this would prevent the AV from recognizing them as js during the file creation hook.

Edit:
Meteor does indeed store a copy of all packages that get installed on any project into C:\Users\rozzzly\AppData\Local\.meteor\packages (substitute rozzzly with your own username of course) @stubailo I'm not sure who could best answer this, but what's the reasoning behind (in linux too) keeping a lot of the same content in appdata / ~/.meteor (on linux) and a .meteor/local on a per project? Just so package dep resolver doesn't have to redownload everything as often, correct? it might turn out that it would be beneficial to move this to C:\Users\rozzzly\.meteor as its not always targeted by AVs in the same way. We'll have to see...

@running-coder what versions of node are you using at work/home? and are you using CMD, mingw, cgywin, etc to run your meteor run command?

[running-coder]

I'm launching Meteor from the CMD prompt without admin privileges on both Work/Home setups. Both have the project directory located on C:\Meteor\MyApp and are located on the Windows-installed drive (no samba shenanigans with mounted drives).
Work:
Win7, nodejs v0.12.0, updated Microsoft Framework to 4 (without such update I was unable to install VM guess additions on another project because the cmd prompt wouldn't allow it)
Home:
Win 10.0.10240, nodejs v5.1.0, Win Framework 4.5.50938

Let me know if you need more info

[rozzzly]

@running-coder First off, thanks for the detailed response. 👍

wait...

You're using node v5.1.0?!?!?!

Meteor support is locked at v0.12.x (@see #5124) right now... I'm just surprised that it doesn't crash and burn giving you some error. That's what it should do. I don't know how it'd be possible for it to run. Did you download the windows installer from the homepage? Could that have installed a different, supported version of node? I've had it happen where my node is symlinked to the newest version but meteor or some other software is actually using the absolute path of different version which can cause some confusion when you run node -v/node --version and see a different version than what you expected. I tried to see on windoze my system if I could find where multiple versions of node are stored, but I develop over ssh to my VPS so I don't think I've ever updated node? 😟 I can't find folder containing different versions. I don't get yum so I can't really quickly test this... I'll have to wait for some others to weigh in.

Also what if meteor is running on node v5.1.0?? it should fail upon execution... Has anyone thought to test this? is this windows specific? if so why? what about node on windows is causing it to still be able to work? Is there some crazy backwards compatiblity going on where 5 tries to emulate v0.12.0? that would explain the performance drop. Other than that, I don't know what to think. AVs still seem to be part of the problem (why else would we see the spike on AVG) but we should figure this out first. I have a few other theories, but we need to dig deeper.

edit:
I stand corrected. C:\Users\rozzzly\AppData\Local\.meteor\packages\meteor-tool\1.1.4\mt-os.windows.x86_32\dev_bundle\bin holds my node. 🤦 Well with that off the table, I would just say you should compare your AV setup at home/work. Both AVG? what about versions, policies, etc... that all could make a difference. It's probably going to be one of the toggleable modules titled "runtime security"/"active scanning"/"real time defense"/etc. I'm going to make the assumption that on your work computer you don't necessarily have control over those factors. You're sysadmin would be the guy to go to I guess.

[stubailo]

@rozzzly Meteor on Windows, just like Meteor on Linux and Mac, has its own hidden version of Node, so it doesn't interact with the globally installed Node at all. The version of Node installed on the system shouldn't affect anything related to Meteor.

[fabiodr]

Wait, this 1 second is for a browser refresh or a Meteor recompilation after some file edit @running-coder?
If it is for compilation, i must know how i can get it too, and i believe a lot of people too. An output of cmd with build profiler enabled would help too.

[ozsay]

A temporary solution

I added http://localhost:3000/ to the exceptions list in AVG (advanced settings).

It reduced the the refreshing time from 10~ seconds to 1~ second.

[running-coder]

@fabiodr The ~1-3 sec is from the time I save a file (project recompiles) and the final display on the browser after the refresh, so the whole process.

@ozsay Glad to hear it helped out!

[fabiodr]

I tried everything... but it is far from ~3 seconds.

Can someone post the output of the command below on a new Meteor project or Wekan, just for comparison? @running-coder @ozsay

set METEOR_PROFILE=50 & meteor

Here are my timings for a new project, editing the .js file in Win 8.1 64bits / i7 950 3Ghz / 6Gb RAM / SSD:

=> App running at: http://localhost:3000/
Type Control-C twice to stop.

| Selecting package versions |
| Select Package Versions: 2711.1
| Input#loadFromCatalog (sqlite): 2617.6
| other Select Package Versions: 89.7
|
| Input#loadFromCatalog (sqlite): 2617.6
| other Select Package Versions: 89.7
| Total: 2707.4
| Building the application
| files.writeFile: 81.5
| Rebuild App: 1636.6
| bundler.bundle..makeClientTarget: 219.5
| Target#make: 219.4
| Target#_runCompilerPlugins: 103.8
| processing files with ecmascript (for target web.browser): 74.2
| other processing files with ecmascript (for target web.browser): 74.1
| Target#_emitResources: 84.7
| PackageSourceBatch#getResources: 80.5
| PackageSourceBatch#_linkJS: 78.8
| bundler.bundle..makeServerTarget: 160.5
| Target#make: 160.3
| Target#_emitResources: 106.9
| PackageSourceBatch#getResources: 104.1
| PackageSourceBatch#_linkJS: 102.5
| other PackageSourceBatch#_linkJS: 66.4
| bundler..writeSiteArchive: 1237.0
| bundler..writeTargetToPath: 1089.0
| ClientTarget#write: 506.6
| Builder#write: 155.1
| files.writeFile: 110.6
| bundler..writeFile: 255.0
| Builder#write: 254.2
| files.writeFile: 212.7
| other ClientTarget#write: 93.3
| ServerTarget#write: 575.5
| JsImage#write: 535.9
| Builder#write: 443.7
| files.writeFile: 340.2
| other JsImage#write: 88.1
| Builder#complete: 138.3
| files.rm_recursive: 137.4
|
| files.writeFile: 762.8
| files.rm_recursive: 138.4
| other PackageSourceBatch#_linkJS: 114.6
| other ClientTarget#write: 93.3
| files.rename: 91.6
| other JsImage#write: 88.1
| other processing files with ecmascript (for target web.browser): 74.1
| files.readFile: 54.1
| other Builder#write: 51.4
| Total: 1468.5
=> Meteor server restarted

sqlite seems too slow..

Thanks

[ozsay]

@fabiodr

I profiled the server initial startup, client modified and server modified.

Win 7 64bit SP1 | meteor 1.2.1 | i5 750 2.6Ghz | 8gb RAM | HDD

=> Started proxy.
| Selecting package versions
| JsImage#load: 368.9
| other JsImage#load: 367.3
| Select Package Versions: 3456.9
| Input#loadFromCatalog (sqlite): 3126.3
| Solver#getAnswer: 306.7
| new Logic.Solver (MiniSat start-up): 56.2
| minimize total_packages: 151.7
|
| Input#loadFromCatalog (sqlite): 3126.3
| other JsImage#load: 367.3
| minimize total_packages: 151.7
| new Logic.Solver (MiniSat start-up): 56.2
| Total: 3701.6
=> Started MongoDB.
| Building the application |
| files.readFile: 1979.2
| JsImage#load: 790.1
| other JsImage#load: 784.3
| files.writeFile: 16135.3
| files.rename: 9231.8
| Rebuild App: 17405.3
| compiler.compile(the app): 4189.9
| Isopack#ensurePluginsInitialized: 4156.8
| JsImage#load: 2667.9
| files.exists: 924.0
| other JsImage#load: 1743.9
| other Isopack#ensurePluginsInitialized: 1488.9
| bundler.bundle..makeClientTarget: 7735.6
| Target#make: 7735.4
| Target#_runCompilerPlugins: 592.1
| processing files with ecmascript (for target web.browser): 50
.1
| other processing files with ecmascript (for target web.br
wser): 507.9
| Target#_emitResources: 7037.3
| PackageSourceBatch#getResources: 7032.3
| PackageSourceBatch#_linkJS: 7030.0
| files.readFile: 133.5
| linker.fullLink: 6685.6
| linker Module#getPrelinkedFiles: 2555.9
| linker File#getPrelinkedOutput: 715.9
| getPrelinkedFiles toStringWithSourceMap (pack
ges): 1701.0
| other linker Module#getPrelinkedFiles: 137.4
| linker Module#computeAssignedVariables: 4083.0
| linker File#computeAssignedVariables: 4080.1
| other PackageSourceBatch#_linkJS: 205.4
| ClientTarget#minifyCss: 58.0
| bundler.bundle..makeServerTarget: 4380.9
| Target#make: 4380.7
| Target#_runCompilerPlugins: 923.3
| processing files with ecmascript (for target os.windows.x86_3
): 895.0
| other processing files with ecmascript (for target os.win
ows.x86_32): 894.7
| Target#_emitResources: 3437.0
| PackageSourceBatch#getResources: 3431.7
| PackageSourceBatch#_linkJS: 3428.7
| files.readFile: 104.7
| linker.fullLink: 3189.1
| linker Module#getPrelinkedFiles: 2145.5
| linker File#getPrelinkedOutput: 588.3
| getPrelinkedFiles toStringWithSourceMap (pack
ges): 1427.0
| other linker Module#getPrelinkedFiles: 130.2
| linker Module#computeAssignedVariables: 1002.1
| linker File#computeAssignedVariables: 999.3
| other PackageSourceBatch#_linkJS: 129.9
| bundler..writeSiteArchive: 1083.7
| bundler..writeTargetToPath: 1023.8
| ClientTarget#write: 225.1
| Builder#write: 79.4
| bundler..writeFile: 94.6
| Builder#write: 93.5
| Builder#complete: 462.7
| files.rename: 462.4
| ServerTarget#write: 328.5
| JsImage#write: 297.5
| Builder#write: 191.5
| files.writeFile: 62.4
| files.rename: 53.2
| other JsImage#write: 97.5
|
| files.writeFile: 16306.7
| files.rename: 9820.4
| linker File#computeAssignedVariables: 5079.5
| getPrelinkedFiles toStringWithSourceMap (packages): 3128.0
| other JsImage#load: 2528.2
| files.readFile: 2243.4
| other Isopack#ensurePluginsInitialized: 1488.9
| linker File#getPrelinkedOutput: 1304.3
| files.exists: 964.1
| other processing files with ecmascript (for target os.windows.x86_32): 894.7
| other processing files with ecmascript (for target web.browser): 507.9
| other PackageSourceBatch#_linkJS: 335.4
| other linker Module#getPrelinkedFiles: 267.5
| other JsImage#write: 97.5
| other linker.fullLink: 88.1
| files.stat: 68.2
| ClientTarget#minifyCss: 58.0
| other Builder#write: 55.7
| Total: 45236.6
=> Started your app.

=> App running at: http://localhost:3000/
Type Control-C twice to stop.

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|

=> Client modified -- refreshing
| Selecting package versions |
| Select Package Versions: 2592.1
| Input#loadFromCatalog (sqlite): 2502.4
| other Select Package Versions: 82.8
|
| Input#loadFromCatalog (sqlite): 2502.4
| other Select Package Versions: 82.8
| Total: 2585.2
| Building the application -
| files.writeFile: 114.3
| Rebuild App: 1496.0
| bundler.bundle..makeClientTarget: 135.8
| Target#make: 135.7
| Target#_runCompilerPlugins: 73.2
| bundler.bundle..makeServerTarget: 90.9
| Target#make: 90.8
| bundler..writeSiteArchive: 1225.5
| bundler..writeTargetToPath: 1020.4
| ClientTarget#write: 310.3
| Builder#write: 104.1
| bundler..writeFile: 108.4
| Builder#write: 106.9
| other ClientTarget#write: 95.6
| Builder#complete: 198.1
| files.rename: 197.9
| ServerTarget#write: 504.6
| JsImage#write: 461.4
| Builder#write: 299.4
| files.writeFile: 150.2
| files.rename: 64.6
| other JsImage#write: 150.0
| Builder#complete: 188.0
| files.rm_recursive: 156.0
|
| files.rename: 365.7
| files.writeFile: 358.1
| files.rm_recursive: 158.0
| other JsImage#write: 150.0
| other ClientTarget#write: 95.6
| files.stat: 93.1
| other Builder#write: 61.8
| other Target#_runCompilerPlugins: 50.8
| Total: 1333.0
=> Meteor server restarted

[running-coder]

@fabiodr
Windows 7 SP1 | i7-4790 | 16gb ram | ssd

I've created a new app using meteor create myapp ran from windows cmd without admin privileges, here is the dump (the first is from the initial build, the second is from the subsequents):

initial build (kinda slow)
C:\Meteor\myapp>set METEOR_PROFILE=50 & meteor
[[[[[ C:\Meteor\myapp ]]]]]

=> Started proxy.
=> Started MongoDB.
| Selecting package versions /
| JsImage#load: 486.3
| other JsImage#load: 484.2
| Select Package Versions: 2404.0
| Input#loadFromCatalog (sqlite): 2192.1
| Solver#getAnswer: 196.0
| minimize total_packages: 106.7
|
| Input#loadFromCatalog (sqlite): 2192.1
| other JsImage#load: 484.2
| minimize total_packages: 106.7
| files.readFile: 54.9
| Total: 2837.9
| Building the application /
| bundler.readJsImage: 94.8
| files.readFile: 91.0
| JsImage#load: 812.9
| other JsImage#load: 807.1
| files.writeFile: 16404.9
| files.rename: 5591.1
| Rebuild App: 14456.7
| compiler.compile(the app): 5134.7
| Isopack#ensurePluginsInitialized: 5121.5
| JsImage#load: 2931.8
| files.exists: 1088.5
| other JsImage#load: 1843.3
| other Isopack#ensurePluginsInitialized: 2189.7
| bundler.bundle..makeClientTarget: 6103.2
| Target#make: 6103.1
| Target#_runCompilerPlugins: 373.3
| processing files with ecmascript (for target web.browser): 308.1
| other processing files with ecmascript (for target web.browser): 308.0
| Target#_emitResources: 5612.2
| PackageSourceBatch#getResources: 5609.3
| PackageSourceBatch#_linkJS: 5607.7
| files.readFile: 88.2
| linker.fullLink: 5374.6
| linker Module#getPrelinkedFiles: 1699.7
| linker File#getPrelinkedOutput: 477.8
| getPrelinkedFiles toStringWithSourceMap (packages): 1111.0
| other linker Module#getPrelinkedFiles: 110.0
| linker Module#computeAssignedVariables: 3631.8
| linker File#computeAssignedVariables: 3629.2
| other PackageSourceBatch#_linkJS: 140.9
| ClientTarget#minifyCss: 88.7
| bundler.bundle..makeServerTarget: 2723.2
| Target#make: 2723.1
| Target#_runCompilerPlugins: 558.4
| processing files with ecmascript (for target os.windows.x86_32): 542.3
| other processing files with ecmascript (for target os.windows.x86_32): 542.1
| Target#_emitResources: 2152.5
| PackageSourceBatch#getResources: 2149.1
| PackageSourceBatch#_linkJS: 2147.9
| files.readFile: 58.6
| linker.fullLink: 1988.4
| linker Module#getPrelinkedFiles: 1388.5
| linker File#getPrelinkedOutput: 372.9
| getPrelinkedFiles toStringWithSourceMap (packages): 931.0
| other linker Module#getPrelinkedFiles: 84.6
| linker Module#computeAssignedVariables: 576.1
| linker File#computeAssignedVariables: 574.1
| other PackageSourceBatch#_linkJS: 97.8
| bundler..writeSiteArchive: 487.4
| bundler..writeTargetToPath: 456.2
| ClientTarget#write: 166.4
| Builder#write: 68.7
| bundler..writeFile: 68.7
| Builder#write: 68.0
| ServerTarget#write: 246.9
| JsImage#write: 217.6
| Builder#write: 158.4
| files.writeFile: 70.0
|
| files.writeFile: 16547.4
| files.rename: 5746.6
| linker File#computeAssignedVariables: 4203.3
| other JsImage#load: 2650.4
| other Isopack#ensurePluginsInitialized: 2189.7
| getPrelinkedFiles toStringWithSourceMap (packages): 2041.9
| files.exists: 1106.0
| linker File#getPrelinkedOutput: 850.8
| other processing files with ecmascript (for target os.windows.x86_32): 542.1
| other processing files with ecmascript (for target web.browser): 308.0
| files.readFile: 255.1
| other PackageSourceBatch#_linkJS: 238.8
| other linker Module#getPrelinkedFiles: 194.6
| ClientTarget#minifyCss: 88.7
| other linker.fullLink: 67.0
| Total: 37030.2
=> Started your app.

=> App running at: http://localhost:3000/
Type Control-C twice to stop.

Terminate batch job (Y/N)?
^C

second build (way faster)
C:\Meteor\myapp>set METEOR_PROFILE=50 & meteor
[[[[[ C:\Meteor\myapp ]]]]]

=> Started proxy.
=> Started MongoDB.
| Selecting package versions |
| JsImage#load: 226.8
| other JsImage#load: 226.2
| Select Package Versions: 2086.0
| Input#loadFromCatalog (sqlite): 1877.0
| Solver#getAnswer: 192.5
| minimize total_packages: 106.2
|
| Input#loadFromCatalog (sqlite): 1877.0
| other JsImage#load: 226.2
| minimize total_packages: 106.2
| Total: 2209.4
| Building the application |
| JsImage#load: 511.4
| other JsImage#load: 510.1
| Rebuild App: 3589.8
| compiler.compile(the app): 2644.9
| Isopack#ensurePluginsInitialized: 2630.7
| JsImage#load: 1746.4
| files.exists: 600.2
| other JsImage#load: 1146.3
| other Isopack#ensurePluginsInitialized: 884.3
| bundler.bundle..makeClientTarget: 191.8
| Target#make: 191.7
| Target#_emitResources: 94.9
| PackageSourceBatch#getResources: 91.9
| PackageSourceBatch#_linkJS: 91.1
| files.readFile: 58.8
| bundler.bundle..makeServerTarget: 139.4
| Target#make: 139.2
| Target#_emitResources: 104.8
| PackageSourceBatch#getResources: 102.2
| PackageSourceBatch#_linkJS: 101.2
| files.readFile: 55.1
| bundler..writeSiteArchive: 606.2
| bundler..writeTargetToPath: 489.4
| ClientTarget#write: 201.6
| Builder#write: 67.3
| bundler..writeFile: 66.9
| Builder#write: 65.9
| other ClientTarget#write: 66.2
| ServerTarget#write: 246.7
| JsImage#write: 209.0
| Builder#write: 156.8
| files.writeFile: 70.5
| Builder#complete: 104.0
| files.rm_recursive: 65.8
|
| other JsImage#load: 1656.3
| other Isopack#ensurePluginsInitialized: 884.3
| files.exists: 614.4
| files.rename: 172.1
| files.writeFile: 138.4
| files.readFile: 135.8
| other PackageSourceBatch#_linkJS: 71.9
| files.rm_recursive: 68.4
| other ClientTarget#write: 66.2
| Total: 3807.7
=> Started your app.

[fabiodr]

Thanks!!

But from the outputs you sent, it is getting more than 5 seconds at least. Not counting the browser refresh.

I installed Ubuntu with dual boot just to see the timing there and i couldn't get less than 6 seconds.

I don't know if anyone can, sqlite takes more than 1.5 sec for each operation no matter how powerfull the computer is, based on the outputs people have sent out there.

[abernix]

This is a duplicate of #5644. Closing in favour of that.