Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JavaScript heap out of memory errors after upgrading 1.7.1 -> 1.8.0 #3679

Closed
jpalo opened this issue Mar 25, 2019 · 54 comments

Comments

Projects
None yet
@jpalo
Copy link

commented Mar 25, 2019

Category

  • Question
  • Typo
  • Bug
  • Additional article idea

Expected or Desired Behavior

No memory errors when building and working with the code in SP Online workbench.

Observed Behavior

Upgraded from 1.7.1 to 1.8.0 and started seeing this in my project. Occurs every 3-4th time I modify file & save, which triggers build. As you can see, when the issue occurs on the copy-assets step, webpack step takes long (1 min in this case, also had it occur just now with 26 secs runtime). When it doesn't fail, webpack step only takes 6-11 secs.

[15:21:39] [tslint] tslint version: 5.12.1
[15:21:39] Starting subtask 'tsc'...
[15:21:39] [tsc] typescript version: 3.3.4000
[15:21:39] Finished subtask 'copy-static-assets' after 179 ms
[15:21:46] Finished subtask 'tsc' after 7.75 s
[15:21:46] Finished subtask 'tslint' after 7.76 s
[15:21:46] Starting subtask 'post-copy'...
[15:21:46] Finished subtask 'post-copy' after 562 μs
[15:21:46] Starting subtask 'collectLocalizedResources'...
[15:21:46] Finished subtask 'collectLocalizedResources' after 1.17 ms
[15:21:46] Starting subtask 'configure-webpack'...
[15:21:46] Finished subtask 'configure-webpack' after 1.15 ms
[15:21:46] Starting subtask 'webpack'...
[15:22:49] Finished subtask 'webpack' after 1.03 min
[15:22:49] Starting subtask 'configure-webpack-external-bundling'...
[15:22:49] Finished subtask 'configure-webpack-external-bundling' after 222 ms
[15:22:49] Starting subtask 'copy-assets'...

<--- Last few GCs --->

[11816:00000191464683D0]   629760 ms: Mark-sweep 1009.4 (1672.0) -> 1009.4 (1671.0) MB, 217.7 / 0.0 ms  allocation failure GC in old space requested
[11816:00000191464683D0]   630004 ms: Mark-sweep 1009.4 (1671.0) -> 1009.4 (1639.5) MB, 243.7 / 0.0 ms  last resort GC in old space requested
[11816:00000191464683D0]   630212 ms: Mark-sweep 1009.4 (1639.5) -> 1009.4 (1639.5) MB, 207.9 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000019448425879 <JSObject>
    1: stringSlice(aka stringSlice) [buffer.js:~555] [pc=000000511DFE3C0C](this=000000ABBE1822D1 <undefined>,buf=0000007CD33D5AB9 <Uint8Array map = 000003C65C2C3A71>,encoding=0000019448435221 <String[4]: utf8>,start=0,end=3488682)
    2: toString [buffer.js:~609] [pc=000000511F3114C8](this=0000007CD33D5AB9 <Uint8Array map = 000003C65C2C3A71>,encoding=0000019448435221 <String[4]: utf8>,start=00...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewRawTwoByteString
 5: v8::internal::Factory::NewStringFromUtf8
 6: v8::String::NewFromUtf8
 7: v8::internal::wasm::AsmType::Extern
 8: std::vector<v8::CpuProfileDeoptFrame,std::allocator<v8::CpuProfileDeoptFrame> >::vector<v8::CpuProfileDeoptFrame,std::allocator<v8::CpuProfileDeoptFrame> >
 9: 000000511E32C466

Package.json

{
  "name": "project-folders",
  "version": "3.1.1",
  "private": true,
  "engines": {
    "node": ">=0.10.0"
  },
  "dependencies": {
    "@microsoft/sp-core-library": "1.8.0",
    "@microsoft/sp-http": "1.8.0",
    "@microsoft/sp-lodash-subset": "1.8.0",
    "@microsoft/sp-property-pane": "1.8.0",
    "@microsoft/sp-webpart-base": "1.8.0",
    "@pnp/common": "~1.2.7",
    "@pnp/logging": "~1.2.7",
    "@pnp/odata": "~1.2.7",
    "@pnp/sp": "~1.2.7",
    "@types/es6-collections": "^0.5.31",
    "@types/es6-promise": "0.0.33",
    "@types/microsoft-ajax": "^0.0.33",
    "@types/react": "16.4.2",
    "@types/react-dom": "16.0.5",
    "@types/sharepoint": "^2016.1.2",
    "@types/webpack-env": "1.13.1",
    "core-js": "^2.5.4",
    "office-ui-fabric-react": "^6.156.0",
    "react": "16.7.0",
    "react-dom": "16.7.0"
  },
  "resolutions": {
    "@types/react": "16.4.2"
  },
  "devDependencies": {
    "@microsoft/sp-build-web": "1.8.0",
    "@microsoft/sp-module-interfaces": "1.8.0",
    "@microsoft/sp-tslint-rules": "1.8.0",
    "@microsoft/sp-webpart-workbench": "1.8.0",
    "@microsoft/rush-stack-compiler-3.3": "0.1.7",
    "@types/chai": "3.4.34",
    "@types/mocha": "2.2.38",
    "ajv": "~5.2.2",
    "gulp": "~3.9.1",
    "webpack-bundle-analyzer": "^3.0.3"
  },
  "scripts": {
    "build": "gulp bundle",
    "clean": "gulp clean",
    "test": "gulp test"
  }
}

tslint.json

{
  "extends": "@microsoft/sp-tslint-rules/base-tslint.json",
  "rules": {
    "class-name": false,
    "export-name": false,
    "forin": false,
    "label-position": false,
    "member-access": true,
    "no-arg": false,
    "no-console": false,
    "no-construct": false,
    "no-duplicate-variable": true,
    "no-eval": false,
    "no-function-expression": true,
    "no-internal-module": true,
    "no-shadowed-variable": true,
    "no-switch-case-fall-through": true,
    "no-unnecessary-semicolons": true,
    "no-unused-expression": true,
    "no-use-before-declare": true,
    "no-with-statement": true,
    "semicolon": true,
    "trailing-comma": false,
    "typedef": false,
    "typedef-whitespace": false,
    "use-named-parameter": true,
    "variable-name": false,
    "whitespace": false
  }
}

tsconfig.json

{
  "extends": "./node_modules/@microsoft/rush-stack-compiler-3.3/includes/tsconfig-web.json", 
  "compilerOptions": {
    "target": "es5",
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "moduleResolution": "node",
    "jsx": "react",
    "declaration": true,
    "sourceMap": true,
    "experimentalDecorators": true,
    "skipLibCheck": true,
    "outDir": "lib",
    "strictNullChecks": false,
    "inlineSources": false,   
    "typeRoots": [
      "./node_modules/@types",
      "./node_modules/@microsoft"
    ],
    "types": [
      "webpack-env",
      "microsoft-ajax",
      "sharepoint"
    ],
    "lib": [
      "es6",
      "dom",
      "es5",
      "es2015.collection"
    ]
  },
  "include": [
    "src/**/*.ts"
  ],
  "exclude": [
    "node_modules",
    "lib"
  ]
}

Steps to Reproduce

  1. Modify any file in project, save file
  2. Auto build fires
  3. Every 3-4th time, gives the error

Project contains 1 React-based web part.

I understand this is a bit vague, apologies, but I don't have much idea what to try, or what kind of addition information you would really need to be able to comment on the issue.

@msft-github-bot

This comment has been minimized.

Copy link
Collaborator

commented Mar 25, 2019

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

@seriewe

This comment has been minimized.

Copy link

commented Mar 27, 2019

We have the same problem.

<--- Last few GCs --->
[14092:0000027560E69E70]   417467 ms: Mark-sweep 1339.7 (1468.8) -> 1339.6 (1469.3) MB, 1244.5 / 0.0 ms  allocation failure GC in old space requested
[14092:0000027560E69E70]   418437 ms: Mark-sweep 1339.6 (1469.3) -> 1339.6 (1426.3) MB, 969.9 / 0.0 ms  last resort GC in old space requested
[14092:0000027560E69E70]   419427 ms: Mark-sweep 1339.6 (1426.3) -> 1339.6 (1426.3) MB, 989.2 / 0.0 ms  last resort GC in old space requested
<--- JS stacktrace --->

==== JS stack trace =========================================
Security context: 00000200413A57C1 <JSObject>
    1: /* anonymous */ [C:\Repos\[MYSOLUTIONNAME]\node_modules\source-map\lib\source-node.js:~342] [pc=000003641B16BB55](this=0000008B8008C211 <JSGlobal Object>,chunk=000003CC049E9C69 <String[1]: }>,original=000000A52B208441 <Object map = 0000002368E4C971>)
    2: SourceNode_walk [C:\Repos\[MYSOLUTIONNAME]\node_modules\source-map\lib\source-node.js:~221] [pc=000003641B16537E](t...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewUninitializedFixedArray
 5: v8::internal::WasmDebugInfo::SetupForTesting
 6: v8::internal::interpreter::BytecodeArrayRandomIterator::UpdateOffsetFromIndex
 7: 000003641A2843C1

Our Setup/Config:
We have approx. 33 WebParts and 5 extensions.

@microsoft/rush-stack-compiler-3.3
"office-ui-fabric-react": "^6.159.1"
...

What we've tried out so far:

  1. Changing the typescript rush-stack-compilers to a lower version does not change anything about the behavior.
  2. After a downgrade "office-ui-fabric-react": "5.131.0" we did not get the error.
  3. If we reduce the number of WebParts we bundle, the error does not occur anymore.
  4. observing the memory with memwatch provides this values:
» full     trend     current         min         max                                                                                                                                                         
» 1            0    92.24 MB   122.19 MB   122.19 MB 
peak during copy-assets
» 29           2     1.16 GB   122.19 MB     1.49 GB
after first reload
» 32         2.2   767.34 MB   122.19 MB     1.49 GB
[14:35:26] Starting subtask 'webpack'...
» 232          0   798.53 MB   122.19 MB     1.49 GB
» 233          0   825.90 MB   122.19 MB     1.49 GB
» 237        0.2   958.69 MB   122.19 MB     1.49 GB
CRASH
<--- Last few GCs --->

Also we see that the subtask webpack takes more than 1min, and our "rebuild" after a change on running gulp serve takes over 2 Minutes.

What is the recommendation for projects with many WebParts, should we move them into several bundles or own projects?

Hope this information helps and it can be improved soon

@jpalo

This comment has been minimized.

Copy link
Author

commented Mar 27, 2019

It doesn't appear to be directly related to number of web parts, or size of the bundle as I only have 1 web part. Considering my webpack subtask runtime (~8 secs in our case), I'd say my project is still very much smaller compared to yours - yet same issue is occurring.

What would be interesting to know is if this is just issue occurring during development time, so is it safe to deploy the solution.

@justdevelopment

This comment has been minimized.

Copy link

commented Mar 27, 2019

We have the same issue, but it fails on the webpack step. When downgrading to v5 it bundles fine but we'd like to use the latest. Seems like maybe there's a loop somewhere just eating memory? There are more issues with the fabric for react package in relation to SPFx, but upgrading typescript seems to solve those (ie #3622). Could it be related to #3618 (comment), where pat miller says internally they are building with typescript 2?

@OliverZeiser

This comment has been minimized.

Copy link

commented Mar 27, 2019

I am seeing the same issue. It seems to be related to using office-ui-fabric-react v6. But why v6 and SPFx don't work together...I can't figure that out

@justdevelopment

This comment has been minimized.

Copy link

commented Mar 27, 2019

Fun fact, if you increase the max memory for node it will bundle/serve eventually.

Using this to set the memory to a max of around 4gb, it works for me:
gulp bundle --max_old_space_size=4000
Using the flag same for gulp serve will allow me to debug:
gulp serve --nobrowser --max_old_space_size=4000

For our project it seems to take around 3.5 gb of memory to complete it's tasks. Does make me wonder why it's using to much memory for the new fabric...?

@jpalo

This comment has been minimized.

Copy link
Author

commented Mar 29, 2019

It peaks at around 3.2 GB in my case at the webpack stage, also CPU usage is constant 30%. Have now waited for few minutes it to complete.

image

And turns out it even crashes with 4000 space_size, but not nearly as often (this is the first time in 2 days).

[12:59:47] Starting subtask 'webpack'...

<--- Last few GCs --->

[20100:000001E25EC267F0] 14003990 ms: Mark-sweep 2740.0 (4863.3) -> 2740.0 (4863.3) MB, 358.3 / 0.0 ms  allocation failure scavenge might not succeed
[20100:000001E25EC267F0] 14004339 ms: Mark-sweep 2740.0 (4863.3) -> 2740.0 (4847.3) MB, 348.1 / 0.0 ms  last resort GC in old space requested
[20100:000001E25EC267F0] 14004686 ms: Mark-sweep 2740.0 (4847.3) -> 2740.0 (4847.3) MB, 347.0 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000208F9C258B9 <JSObject>
    1: DoJoin(aka DoJoin) [native array.js:~94] [pc=0000023FABDA738C](this=000002A3B15022D1 <undefined>,o=000002EF2C15D421 <JSArray[808]>,p=808,D=000002A3B1502371 <true>,z=00000208F9C7CDE9 <String[1]:  >,y=000002A3B15023E1 <false>)
    2: Join(aka Join) [native array.js:~119] [pc=0000023FAB34949E](this=000002A3B15022D1 <undefined>,o=000002EF2C15D421 <JSArray[808]>,p=808,z=00000208F9C7CDE9 <Str...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewRawTwoByteString
 5: v8::internal::AsmJsScanner::IsNumberStart
 6: 0000023FAB2043C1
@GuthrieS

This comment has been minimized.

Copy link

commented Mar 29, 2019

We are also having a similar problem. The solution has been upgraded from SPFx 1.7.0 to version 1.8.0 however, in our case this error appears when it is performing the tslint subtask. This occurs with gulp build, serve or bundle:

[14:24:33] Starting 'build'...
[14:24:33] Starting subtask 'configure-sp-build-rig'...
[14:24:33] Finished subtask 'configure-sp-build-rig' after 5.19 ms
[14:24:33] Starting subtask 'pre-copy'...
[14:24:33] Finished subtask 'pre-copy' after 64 ms
[14:24:33] Starting subtask 'copy-static-assets'...
[14:24:33] Starting subtask 'sass'...
[14:24:33] Finished subtask 'copy-static-assets' after 175 ms
[14:24:33] Finished subtask 'sass' after 275 ms
[14:24:33] Starting subtask 'tslint'...
[14:24:34] [tslint] tslint version: 5.11.0
[14:24:34] Starting subtask 'tsc'...
[14:24:34] [tsc] typescript version: 2.7.2
[14:24:38] Finished subtask 'tsc' after 4.3 s
[14:24:44] Error - [tslint] FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
[14:24:44] Error - [tslint] 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewUninitializedFixedArray
 5: v8::internal::WasmDebugInfo::SetupForTesting
 6: v8::internal::interpreter::BytecodeArrayRandomIterator::UpdateOffsetFromIndex
 7: 000000BFDEA043C1
[14:24:44] Finished subtask 'tslint' after 12 s
[14:24:44] Starting subtask 'post-copy'...
[14:24:44] Finished subtask 'post-copy' after 782 μs
[14:24:44] Finished 'build' after 12 s
[14:24:45] ==================[ Finished ]==================

By removing a lot of the rules from the tslint.json file the problem goes away however we know this isn't the solution and the underlying problem is elsewhere. I can confirm similarities as well with the memory issues being described by @jpalo

@andrewconnell

This comment has been minimized.

Copy link
Collaborator

commented Mar 29, 2019

This smells like a common issue due to large projects, such as SPFx+Fabric React v6 (which is now supported with SPFx v1.8). As suggested above, try building with the --max_old_space_size flag & increasing the amount of available memory? For instance:

gulp bundle --max_old_space_size=8192 --ship
@vman

This comment has been minimized.

Copy link
Contributor

commented Mar 30, 2019

Yup we have this issue for one of larger projects as well when upgrading to 1.8 and TypeScript 3.3.4000

Does this have something to do with multiple versions of TypeScript being present in the build setup now? And hence being loaded in memory during bundling? I have noticed the build system still uses different TS versions for different internal packages and and top of that we can use another TS version in our webparts.

(The disk size of node_modules has also increased from ~450mb to ~650mb due to the multiple TS versions)

@OliverZeiser

This comment has been minimized.

Copy link

commented Mar 30, 2019

@andrewconnell it does help increasing the memory but it seems to be some sort of leak. If you are running gulp serve and make changes on your code it will build over and over until it reaches the memory limit. Besides the fact that the build time goes up by about 400% every time. So this is definitley not a solution, but can help to at least build the project once. Adding @patmill and @VesaJuvonen since this seems to be an issue for many of us. I am sure they can add the right people to get a fix/solution for that.

@wictorwilen

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2019

I have similar issues (slower builds for each build and eventually a mem leak) in the Teams generator with nodemon/webpack, that we likely think is related to the ts-loader.

@VesaJuvonen

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2019

We are looking into this internally and will be updating this issue as soon as possible when we have additional things to report.

@zumiek

This comment has been minimized.

Copy link

commented Apr 3, 2019

Same problem here, but at copy-assets:

`[12:49:33] Finished subtask 'configure-webpack-external-bundling' after 962 μs
[12:49:33] Starting subtask 'copy-assets'...

<--- Last few GCs --->

[3928:0000022DD5434290] 69925 ms: Mark-sweep 1228.0 (1495.5) -> 1227.9 (1492.5) MB, 277.9 / 0.0 ms allocation failure GC in old space requested
[3928:0000022DD5434290] 70216 ms: Mark-sweep 1227.9 (1492.5) -> 1227.7 (1446.0) MB, 290.8 / 0.0 ms last resort GC in old space requested
[3928:0000022DD5434290] 70508 ms: Mark-sweep 1227.7 (1446.0) -> 1227.7 (1431.5) MB, 291.5 / 0.0 ms last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000011ADFB258B9
1: stringSlice(aka stringSlice) [buffer.js:~555] [pc=000002DE8F1D9AAF](this=0000009F150022D1 ,buf=000003D72BD5D061 ,encoding=0000011ADFB352A1 <String[4]: utf8>,start=0,end=1853394)
2: toString [buffer.js:~609] [pc=000002DE8F8A298C](this=000003D72BD5D061 ,encoding=0000011ADFB352A1 <String[4]: utf8>,start=00...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node_module_register
2: v8::internal::FatalProcessOutOfMemory
3: v8::internal::FatalProcessOutOfMemory
4: v8::internal::Factory::NewRawTwoByteString
5: v8::internal::Factory::NewStringFromUtf8
6: v8::String::NewFromUtf8
7: v8::internal::wasm::AsmType::Extern
8: std::vector<v8::CpuProfileDeoptFrame,std::allocatorv8::CpuProfileDeoptFrame >::vector<v8::CpuProfileDeoptFrame,std::allocatorv8::CpuProfileDeoptFrame >
9: 000002DE8F8A1386`

@c-eiser13

This comment has been minimized.

Copy link

commented Apr 4, 2019

Same problem as well during webpack step:
OutOfMemoryException

What is interesting for me is I went through the upgrade steps provided by Office 365 CLI and began serving the project successfully yesterday evening. This morning, I removed package-lock.json and node_modules and re-ran npm install. After performing those steps is when this error began happening for me. Project contains 12 web parts and 2 extensions. Adding --max-old-space-size=4000 to serve command allows me to debug

@peterremote1980

This comment has been minimized.

Copy link

commented Apr 13, 2019

this gulp bundle --max_old_space_size=8192 --ship works for me, thanks

@patmill

This comment has been minimized.

Copy link
Contributor

commented Apr 15, 2019

Looking into this a bunch, and have the same results. Namely, with TS3.x I get this a lot more. I've found that bypassing some tslint errors can make it go away as well, but that seems like a bad trade-off. The --max_old_space_size is the best solution. We're going to work on a way to make that easier to use (the latest sp-starter-kit package.json file has the commands as short-cuts.). Unfortunately we can't update the gulp file, because it is too late at that point. So, nothing really to add here, other than to say "we're working on making this less of a problem".

@vman

This comment has been minimized.

Copy link
Contributor

commented Apr 15, 2019

Thanks @patmill, so do you mean this is not some sort of memory leak as we previously thought and just a result of the toolchain needing more memory now for large projects?

@patmill

This comment has been minimized.

Copy link
Contributor

commented Apr 15, 2019

There could still be some sort of tooling leak, or possibly when you live updates the node process lives on longer w/ more memory consumption. I was finding that build sp-starter-kit would flat out fail from the get-go. Will continue to dig in further, but if the community has findings of their own, that would certainly help.

@OliverZeiser

This comment has been minimized.

Copy link

commented Apr 16, 2019

@patmill There is a way better workaround. Using --max_old_space_size does work, but the time it takes for webpack in our larger solution (30+ webparts/extensions) is over 4 minutes compared to under 1 minute with spfx 1.7 and office-ui-fabric-react 5.x

The issue is clearly related to office-ui-fabric-react. So what I did is adding office-ui-fabric-react as external dependency in config.json and everything works fine again.

BUT: This is not a supported as we all know. So i created a custom bundle of office-ui-fabric-react as AMD module with an own namespace and a custom npm alias and so on to prevent collision with the things Microsoft is using internaly and everything is faster and better than ever before ;)

If you would provide an official way to support office-ui-fabric-react as an external dependency of our project, all issues would be solved. In our case the webpack task is now down to 12 seconds....compared to over 4 minutes, that is huge! :)
Maybe you can think about providing an official spfx office-ui-fabric-react build for every SPFx version or something like that.

@andrewconnell

This comment has been minimized.

Copy link
Collaborator

commented Apr 16, 2019

@OliverZeiser said:

If you would provide an official way to support office-ui-fabric-react as an external dependency of our project, all issues would be solved.

Haven't they already done that with the library component type (granted its dev preview now)? https://docs.microsoft.com/en-us/sharepoint/dev/spfx/library-component-overview

@OliverZeiser

This comment has been minimized.

Copy link

commented Apr 16, 2019

@andrewconnell it is not that easy to convert the office-ui-fabric-react package to an spfx library bundle. I have tried that as well but ran into too many issues. The office-ui-fabric-react has some very special build processes that kick in during a build and I did not have the time to dig in any deeper. Also I think it needs webpack 4 where SPFx is still using webpack 3? But if someone would figure out how to make an spfx library package out of the OUIFR repo it could indeed work. And would be a very nice solution.
If you build a single webpart/extension that uses a few componentes from OUIFR it is okay to include those into the bundle (using the --max_old_space_size). But in large solutions it leads to extremly large bundle sizes and very long build times. So it makes sense to externalize OUIFR.

@OliverZeiser

This comment has been minimized.

Copy link

commented Apr 16, 2019

@andrewconnell adding to links regarding the support statements:
#2449
#1344

@patmill

This comment has been minimized.

Copy link
Contributor

commented Apr 16, 2019

Sorry Oliver - one more question. What version of rush stack compiler are you using, and what do your tslint and tsconfig files look like for the memory issue repro?

@OliverZeiser

This comment has been minimized.

Copy link

commented Apr 16, 2019

@patmill We are using "typescript": "3.3.4000" and "@microsoft/rush-stack-compiler-3.3": "0.1.6".
Here is the tsconfig:
{ "extends": "./node_modules/@microsoft/rush-stack-compiler-3.3/includes/tsconfig-web.json", "compilerOptions": { "target": "es5", "forceConsistentCasingInFileNames": true, "module": "esnext", "moduleResolution": "node", "jsx": "react", "declaration": true, "sourceMap": true, "experimentalDecorators": true, "skipLibCheck": true, "outDir": "lib", "inlineSources": false, "strictNullChecks": false, "noUnusedLocals": false, "typeRoots": [ "./node_modules/@types", "./node_modules/@microsoft" ], "types": [ "es6-promise", "webpack-env" ], "lib": [ "es5", "dom", "es2015.collection" ] }, "include": [ "src/**/*.ts", "typings" ], "exclude": [ "node_modules", "lib" ] }

And here the tslint:

{ "rulesDirectory": [ "tslint-microsoft-contrib" ], "rules": { "class-name": false, "export-name": false, "forin": false, "label-position": false, "member-access": true, "no-arg": false, "no-console": false, "no-construct": false, "no-duplicate-variable": true, "no-eval": false, "no-function-expression": true, "no-internal-module": true, "no-shadowed-variable": true, "no-switch-case-fall-through": true, "no-unnecessary-semicolons": true, "no-unused-expression": true, "no-use-before-declare": true, "no-with-statement": true, "semicolon": true, "trailing-comma": false, "typedef": false, "typedef-whitespace": false, "use-named-parameter": true, "variable-name": false, "whitespace": false } }

@patmill

This comment has been minimized.

Copy link
Contributor

commented Apr 16, 2019

OK, thanks. Out of curiosity if you move to rsc-2.9 (which should still work with Fabric 6 I believe) do you hit the same issue?

@OliverZeiser

This comment has been minimized.

Copy link

commented Apr 16, 2019

Just tested it and with rsc-2.9 and typescript 2.9.2 I run into the same problem :( ...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

@rabwill

This comment has been minimized.

Copy link

commented Apr 20, 2019

I have a similar issue in one of our solutions which uses handlebars in one webpart (react with handlebars to be specific), the solution worked fine and did not have an issue bundling etc. Once upgraded to 1.8.0 it started to throw error in our build pipelines with the same issue. It goes away locally when I bundle with gulp bundle --max_old_space_size=8192 --ship. This bundling also takes forever :( If I remove the handlebars part from the component, the solution bundles just fine. I have below webpack configuration

build.configureWebpack.mergeConfig({ additionalConfiguration: (generatedConfiguration) => { generatedConfiguration.resolve.alias = { handlebars: 'handlebars/dist/handlebars.min.js' }; generatedConfiguration.module.rules.push({ test: /\.js$/, loader: 'unlazy-loader', exclude: [path.resolve(__dirname, './lib/')] }); generatedConfiguration.node = { fs: 'empty', readline: 'empty' } return generatedConfiguration; } });

The error
`
<--- Last few GCs --->

[2616:000001F548582480] 412447 ms: Mark-sweep 1685.6 (2285.8) -> 1685.6 (2285.8) MB, 278.6 / 0.0 ms allocation failure scavenge might not succeed
[2616:000001F548582480] 412723 ms: Mark-sweep 1685.6 (2285.8) -> 1685.6 (2269.8) MB, 276.0 / 0.0 ms last resort GC in old space requested
[2616:000001F548582480] 412986 ms: Mark-sweep 1685.6 (2269.8) -> 1685.6 (2269.8) MB, 262.8 / 0.0 ms last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000002BE151258B9
1: DoJoin(aka DoJoin) [native array.js:~94] [pc=000003FE6D4D006C](this=000001FB8C1822D1 ,o=000003DA69258DB1 <JSArray[400]>,p=400,D=000001FB8C182371 ,z=000002BE1517CDA9 <String[1]: >,y=000001FB8C1823E1 )
2: Join(aka Join) [native array.js:~119] [pc=000003FE6C95515E](this=000001FB8C1822D1 ,o=000003DA69258DB1 <JSArray[400]>,p=400,z=000002BE1517CDA9 <Str..
.

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node_module_register
2: v8::internal::FatalProcessOutOfMemory
3: v8::internal::FatalProcessOutOfMemory
4: v8::internal::Factory::NewRawTwoByteString
5: v8::internal::AsmJsScanner::IsNumberStart
6: 000003FE6C8043C1 `

@VesaJuvonen

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

This is now fixed with the 1.8.2 release. If you still see similar memory issues, please submit a new issue, so that we can track that properly. Thanks for reporting this earlier and sorry for the inconvenience caused by this one.

@vman

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

Can confirm this is fixed for us now! Thanks 👍

@VesaJuvonen

This comment has been minimized.

Copy link
Contributor

commented May 7, 2019

thx @vman for the confirmation also from your side. We will keep on optimizing the developer experience even further with the upcoming 1.9 and following versions.

@OliverZeiser

This comment has been minimized.

Copy link

commented May 8, 2019

@VesaJuvonen I can NOT confirm that this is working. In our solution i still get the error after upgrading to 1.8.2 and node 10.
If I add office ui fabric react as external dependency, everything works fine. Including it into the bundle leads to this out of memory error.

@GuthrieS

This comment has been minimized.

Copy link

commented May 8, 2019

I can confirm that this issue has been resolved for us also. Thanks guys and great work 👍

@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

I am also still getting the out of memory error after upgrading to 1.8.2. I’ve tried on node 8 and node 10 with same results. For me, I only get the error in a large project, 16 web parts and 2 extensions. If a new issue should be opened, I can do so and provide screens/package.json if needed.

@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

After further testing, by moving OUIFR to external dependency as @OliverZeiser mentioned, the memory error goes away. Also of note, this has reduced my build time from 2.5-3 mins down to 39 seconds.

@patmill

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

Can you give the versions of the build and tooling you are using in your package.json? ie, which rush-stack-compiler, as well as which version of office-ui-fabric-react?

@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

Sure thing, see below and let me know if you need anything further.

package1

package2

I'm using 3.2 of compiler:

tsconfig

@andrewconnell

This comment has been minimized.

Copy link
Collaborator

commented May 8, 2019

You don't want multiple @microsoft/rush-stack-compiler-#-#& package installed.. you only want one. In this case, you only want 3.2.

Try a quick fix of (1) uninstalling the other two, (2) delete node_modules & (3) reinstall all packages with npm install. This will ensure everything is cleaned up.

@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

Thanks @andrewconnell for the suggestion. I followed the steps, removing 2.7 and 3.3, deleting node_modules, then reinstalling everything. I still get the error on both node 8 and 10.

node 8:

outofmem1
outofmem2

node 10

outofmem-10-1
outofmem-10-2

@patmill

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

Let's try a few things:
1 - remove all those extra rush-stack-compiler references.
2 - remove the reference to typescript
3 - remove the reference to the generator
followed by

npm install

If that doesn't work, try using "@microsoft/rush-stack-compiler-2.9": "0.7.7" instead (and update your tsconfig.json file as well)

after this, can you run

npm list webpack
@patmill

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

(weird, my response appears to have not posted).

You've got a lot of extra stuff in your package.json file. Can you remove:
1 - All the extra references to rush-stack-compiler
2 - The reference to generator-sharepoint
3 - the reference to typescript

npm install

then run and respond here with the contents of

npm list webpack

You'll then want to clean / rebuild. If that doesn't work, can you switch your rush-stack-compiler to version -2.9@0.7.7 and go through those steps again? You'll need to update your tsconfig.json file to point to the -2.9 path.

@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

Thanks @patmill my responses and posts are all out of order for some reason. Not sure if you saw my results after removing the additional rush stack compiler entries, but I still get the error. I will try your suggestions as well and report back.

@patmill

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

Wow - the out-of-order things are making my brain hurt. @c-eiser13 - can you give the results of

npm list webpack
@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

Ha, agree @patmill , it is very hard to follow.

webpack

@andrewconnell

This comment has been minimized.

Copy link
Collaborator

commented May 8, 2019

Very strange... GH is posting some comments "in the future". Note the 4 at the bottom of the issue say "X hours from now" where everything else is "X hours ago".

That looks buggy...

@patmill

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

@c-eiser13 - if you are up for it, can you share your project out? If you want, you can send it to me at my user name here @microsoft.com

I used the sp-starter-kit as my stress test for this, and it has pnp, fabric, 17 webparts, 7 extensions, etc. Hopefully we can get a solution for you while we work on the next version of the tools.

@c-eiser13

This comment has been minimized.

Copy link

commented May 8, 2019

Thanks @patmill for the sake of thoroughness, I will complete the steps you mentioned in testing. I only tested removing the 2.7 and 3.3 version of the compiler, I have not tested the other steps you provided. Once I am able to, I will report back and possibly send over to you. To be honest, I can handle adding the max-old-space for the workaround, more importantly, I am hoping a future release will enhance the build times of large projects. Currently while developing, every save I have to wait 1.5-2.5 minutes to see the result. Externalizing OUIFR brings me back to around 30 seconds, so hopefully there will be a way to put this into a library component or gain more control over which components are processed while serving. Thanks again for all your help and responses, and once I'm able to test, I will post back.

@patmill

This comment has been minimized.

Copy link
Contributor

commented May 8, 2019

Thanks. I'm just wanting to sort out where the issues are for other people as well (or just admit that we need the max-old-space and pivot the tooling to accept that). One last question, around the externalizing of OUIFR. What does that actually look like at runtime? Do you load the entire OUIFR js package? I ask because you might be trading off 2 minutes of build for 800+kb of minimized js loaded onto the page. That's an unhealthy trade off. Maybe externalizing during the inner loop and bundling during the final build is a better approach? At any rate, keep us posted.

@c-eiser13

This comment has been minimized.

Copy link

commented May 9, 2019

@patmill i just added OUIFR to the externals section of config.json and was able to gulp serve successfully in 1/4 the time it had been taking. Once I saw this reply, I tried again and actually went to the site, but all web parts were errored out. I believe it was @OliverZeiser that had a solution where he created his own external bundle of OUIFR, for me, it just allowed me to serve with no error and much quicker than previously.

I followed your previous suggestions, removing the generator and typescript from package.json, deleted node_modules, ran npm install, then gulp clean and here are my results.

  1. run gulp serve --nobrowser --> out of memory error
  2. gulp build --> Success
  3. gulp bundle --ship --> Success

Perhaps I should have been more specific, in all of my previous testing, the error was always received when running gulp serve (and still is), i was not trying build or bundle.

I downgraded compiler to 2.9, and the results were the same as above, error in serve and success in build and bundle.

Thanks again for your help and feedback, let me know if there is anything else you would like me to try. I will discuss with my team about sending the solution over to you if that may help find the root cause and let you know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.