Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coredump On Close - "watcher.node"? #136264

Closed
vindicatorr opened this issue Nov 2, 2021 · 28 comments · Fixed by #136569
Closed

Coredump On Close - "watcher.node"? #136264

vindicatorr opened this issue Nov 2, 2021 · 28 comments · Fixed by #136569
Assignees
Labels
confirmed Issue has been confirmed by VS Code Team member electron Issues and items related to Electron file-watcher File watcher freeze-slow-crash-leak VS Code crashing, performance, freeze and memory leak issues insiders-released Patch has been released in VS Code Insiders linux Issues with VS Code on Linux
Milestone

Comments

@vindicatorr
Copy link

Issue Type: Bug

VS Code version: Code - Insiders 1.62.0-insider (ff1e16e, 2021-10-29T07:49:33.336Z)
OS version: Linux x64 5.13.1-dirty
Restricted Mode: No

System Info
Item Value
CPUs Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz (4 x 3391)
GPU Status 2d_canvas: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
oop_rasterization: disabled_off
opengl: enabled_on
rasterization: disabled_software
skia_renderer: enabled_on
video_decode: disabled_software
vulkan: disabled_off
webgl: enabled
webgl2: enabled
Load (avg) 2, 4, 4
Memory (System) 7.23GB (0.24GB free)
Process Argv --crash-reporter-id a2928a5c-190e-4d25-8038-9980bdeb3821
Screen Reader no
VM 0%
DESKTOP_SESSION plasma
XDG_CURRENT_DESKTOP KDE
XDG_SESSION_DESKTOP KDE
XDG_SESSION_TYPE x11
Extensions (14)
Extension Author (truncated) Version
xml Dot 2.5.0
vscode-solution-explorer fer 0.4.3
ilspy-vscode ics 0.12.1
vscode-nuget-package-manager jmr 1.1.6
cortex-debug mar 0.3.7
blazorwasm-companion ms- 1.0.1
csharp ms- 1.23.16
vscode-dotnet-runtime ms- 1.0.0
cmake-tools ms- 1.9.0
cpptools ms- 1.6.0
devicetree plo 0.1.1
vscode-xml-complete rog 0.3.0
msbuild-project-tools tin 0.3.15
cmake twx 0.0.17
A/B Experiments
vsliv695:30137379
vsins829:30139715
vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492:30256197
pythontb:30258533
pythonvspyt551cf:30291413
pythonptprofiler:30281269
vshan820:30294714
pythondataviewer:30285072
pythonvsuse255:30319630
vscod805cf:30301675
pythonvspyt200:30323110
vsccppwt:30382699
bridge0708:30335490
pygetstartedt2:30353727
bridge0723:30353136
pythonrunftest32:30365365
pythonf5test824:30361779
javagetstartedt:30350119
pythonvspyt187:30365360
pydsgst2:30361790
vsaa593cf:30376535

Nov 01 22:02:50 computerName systemd-coredump[265334]: [🡕] Process 65506 (code-insiders) of user 1000 dumped core.
                                                 
                                                 Found module <pathTo>/VSCode-linux-x64/code-insiders with build-id: 1db4b1bb686b67bdfa55c5a786f4efa1ad24cdeb
                                                 Found module <pathTo>/VSCode-linux-x64/resources/app/node_modules.asar.unpacked/@parcel/watcher/build/Release/watcher.node without build-id.
                                                 Found module <pathTo>/VSCode-linux-x64/libffmpeg.so with build-id: 25b3712cab44f7589c54878391ac8deca75485f2
                                                 Found module linux-vdso.so.1 with build-id: ff8c606f703af4cd18720b6d663de3d350aa3acf
                                                 Found module libstdc++.so.6 with build-id: 8ab0e57054dd1dcba681f217016afc6a4e639783
                                                 Found module libicudata.so.69 with build-id: 0ab994a49ef1848499c4af333b3266f28432a922
                                                 Found module libbrotlicommon.so.1 with build-id: f68934f94312f770550ebc6c5acfd6359b098b07
                                                 Found module libgpg-error.so.0 with build-id: ba85170c2d9343ea05eea8fa2048c212ff4ef552
                                                 Found module libdatrie.so.1 with build-id: 6fe3b6ece2c8e7d11869fa051375128d8f808f58
                                                 Found module libxml2.so.2 with build-id: b9441ffabb52b50c290152592f17cd78a381d493
                                                 Found module libjson-glib-1.0.so.0 with build-id: 5865be5fcc01b48a0a113e0145ec45529bff89a6
                                                 Found module libsqlite3.so.0 with build-id: b5c21727dd6a9dcb95e6e57c0767158fddcb3245
                                                 Found module libicui18n.so.69 with build-id: 9cdecde5b2e47a2bd81dc14915cbfefcade76c12
                                                 Found module libicuuc.so.69 with build-id: 5cf18c56e2f64efdac32cf61fb9c0c48e9bb1797
                                                 Found module libstemmer.so.0 with build-id: d3e1b30a80f7b5ead032dd8f97319865366449eb
                                                 Found module libbrotlidec.so.1 with build-id: f871e6f204ab8d48099915126ba01be989a8000f
                                                 Found module libbz2.so.1.0 with build-id: 919597c477c9b2cb9cdbb7745ed6494ac0e6da60
                                                 Found module libgraphite2.so.3 with build-id: 47761dc11e553f519cde97ed9ee985be12ccdae2
                                                 Found module libgcrypt.so.20 with build-id: db45f5d5e0f7af1e77324fea1885f974619ad268
                                                 Found module libcap.so.2 with build-id: c1674f9082fedd415876b9f7d9712269163259b5
                                                 Found module liblz4.so.1 with build-id: e63600ab23b2f6997f42fac2fa56e1f02ce159a1
                                                 Found module libzstd.so.1 with build-id: 4b10444c1560ebc574af4d5f488b7408b22d450e
                                                 Found module liblzma.so.5 with build-id: 8b615460aa230708c5183f16bede67aa0437d95e
                                                 Found module libblkid.so.1 with build-id: f03a24e92c9852272bd0049b5bcb7ed6cf0e664f
                                                 Found module libwayland-server.so.0 with build-id: 7156897eafc4b7562f2b214e6ad8ca6c85c57e47
                                                 Found module libXdmcp.so.6 with build-id: 8ca0792d23c8b8b4c0864297512349292bea5955
                                                 Found module libXau.so.6 with build-id: 1c67764663e07bec24d8951e5fd93f4d165979ff
                                                 Found module libpixman-1.so.0 with build-id: 341f793dcada3a48a306a793d265a517e3f2e7d6
                                                 Found module libxcb-shm.so.0 with build-id: fb797f299a446f559a95afcc168227482cc800d1
                                                 Found module libxcb-render.so.0 with build-id: a37bdb37744b508be9dd29fd9ccb9bb0170d43e5
                                                 Found module libXrender.so.1 with build-id: 97e0b9ab6ba96ebc86527cc2b3c3078aad8616b3
                                                 Found module libthai.so.0 with build-id: 3fdce1a7db2b94d9168eea540dbf5ff5caa3b8ad
                                                 Found module libXinerama.so.1 with build-id: dbb6adece34ea2766d3efcd628af8f9757344dd6
                                                 Found module libXcursor.so.1 with build-id: eb6315bcd7526aaf51b0a1307efdc587cea0a152
                                                 Found module libwayland-egl.so.1 with build-id: 791e39283cdcc5a08cfd091b8cd4c59869a11278
                                                 Found module libwayland-cursor.so.0 with build-id: ca6839778a1490fdd8e3d8aadc9604afa19ab8ea
                                                 Found module libwayland-client.so.0 with build-id: d274361e7aa57c9b63504a039e063322450cbb9c
                                                 Found module libtracker-sparql-3.0.so.0 with build-id: ab1223261f5b835c18018411544c0c7a16b95046
                                                 Found module libcloudproviders.so.0 with build-id: ed1d9c9e60be4e858ad148484767249be1edf95f
                                                 Found module libXi.so.6 with build-id: 16603be937a02a7e61b0b0395d064be7efd86f49
                                                 Found module libepoxy.so.0 with build-id: b6357fc6b748c512f90339433d74502b9d0621dc
                                                 Found module libcairo-gobject.so.2 with build-id: abbf5a8a182ea1b8b7d68540df5c2e25c0bdae0c
                                                 Found module libfribidi.so.0 with build-id: 79124bad061c2aab4ec89f25d067363d5b781114
                                                 Found module libfreetype.so.6 with build-id: 65e7f4a1e5557b0ceeaa433e5356f857fe9c669b
                                                 Found module libfontconfig.so.1 with build-id: 1103a641395c7d3b42e49b793d3a9ea927c77bf6
                                                 Found module libpangoft2-1.0.so.0 with build-id: 52a846673f7bf9e8acdd0ae9a825926c9cfc5920
                                                 Found module libharfbuzz.so.0 with build-id: f954dfb80265c0dd2484e766282305a739b8239d
                                                 Found module libpangocairo-1.0.so.0 with build-id: 2554be0474a6ee311b6e9aa4fd79a244167cdcfd
                                                 Found module libtiff.so.5 with build-id: e8d89a0fb0847641ddb75c5f03fa89d68155a75d
                                                 Found module libjpeg.so.8 with build-id: 5d102b1a7291a621bba049969d25ead0ccecc6cc
                                                 Found module libpng16.so.16 with build-id: 2dc0bce07f199bf983c07a05fb95a6f4af83a9b3
                                                 Found module libsystemd.so.0 with build-id: eb468d8d869a836974a3806872aa2ae83078ed4d
                                                 Found module libplc4.so with build-id: e3d338de1e8d5ea2f98a87af9157b17f92278d83
                                                 Found module libplds4.so with build-id: a2dcd5d4a1d3dccbf591e36bcc8d84162751c0a8
                                                 Found module libresolv.so.2 with build-id: c915c72668282861a813f7ea3c0780f37b681dc0
                                                 Found module libmount.so.1 with build-id: ff9a1d99d35cf640d0bfdaba294854372672c29b
                                                 Found module libz.so.1 with build-id: 81bf6e728a6d6f5b105b0f8b25f6c614ce10452a
                                                 Found module libgmodule-2.0.so.0 with build-id: 34cabbd12d9baf397a0255f174e4c96165bc329b
                                                 Found module libpcre.so.1 with build-id: 845483dd0acba86de9f0313102bebbaf3ce52767
                                                 Found module libffi.so.8 with build-id: f90d8b734f6de9b25faedb8cbfab7054dafc0a42
                                                 Found module ld-linux-x86-64.so.2 with build-id: 040cc3dd10461562f177df39e3be2f3704258c3c
                                                 Found module libc.so.6 with build-id: 4b406737057708c0e4c642345a703c47a61c73dc
                                                 Found module libgcc_s.so.1 with build-id: 7f8508bb914546ada778809b64b99d234337d835
                                                 Found module libatspi.so.0 with build-id: 046ee7a264956ff92020b0773ab583b23b93933c
                                                 Found module libasound.so.2 with build-id: 9288c92ff737b1999761df2018fcd0e7cbf46d5a
                                                 Found module libgbm.so.1 with build-id: 545f14ad495e5b1abc10151280daa3c2d736447d
                                                 Found module libxkbcommon.so.0 with build-id: 1f1bc7527e57c886c3df5803068122e1971d4724
                                                 Found module libxcb.so.1 with build-id: 0d1ef11740a5daad2ee331e812a51aa6574af222
                                                 Found module libexpat.so.1 with build-id: 8850138eae6d9d4d43c5c4b2ac48393bc4279037
                                                 Found module libXrandr.so.2 with build-id: d4e869b5c72541e1de8f96b456248987ea52d51b
                                                 Found module libXfixes.so.3 with build-id: 0a05c7e8714522bfbdd7c0027c3e2a94965664b0
                                                 Found module libXext.so.6 with build-id: d70f24beb4fad748d6becffdcc13e51be0a2ebfa
                                                 Found module libXdamage.so.1 with build-id: a1d867e093c16944b2c0083be98050ef35c9e5c2
                                                 Found module libXcomposite.so.1 with build-id: f1bb5ffb6203ab2907ac044a204ce3902a79f1c1
                                                 Found module libX11.so.6 with build-id: e9192497aa39fab51358826d1a1b5fb77edc5081
                                                 Found module libm.so.6 with build-id: 2b8fd1f869ecab4e0b55e92f2f151897f6818acf
                                                 Found module libcairo.so.2 with build-id: 06820af9cf79c2deb207d9533a61caf04e67a8ad
                                                 Found module libpango-1.0.so.0 with build-id: 5deae7c16592ea404754114e8ae56ddf9cf4ca7e
                                                 Found module libgdk-3.so.0 with build-id: b056619db3ba9fc6b33353703d60298125a23473
                                                 Found module libgtk-3.so.0 with build-id: f29f0b8d0a86f1371a88cd1484229c1f31a2b122
                                                 Found module libgdk_pixbuf-2.0.so.0 with build-id: ca4beb15a30de8d8767f7bca8222038c61b58b36
                                                 Found module libdrm.so.2 with build-id: 3aeff5403ca8d7589eabc05752eb613937f454a1
                                                 Found module libdbus-1.so.3 with build-id: 74f2ab9c60512f3a93c932c3f627564d42e0b11e
                                                 Found module libatk-bridge-2.0.so.0 with build-id: 1161ebbcc5aa791075dfa026671875e5417287bd
                                                 Found module libatk-1.0.so.0 with build-id: aacaa40363d304858b654728a893bc212a4ca119
                                                 Found module libnspr4.so with build-id: d737bfb932206eb128a493cc70cedd63714e814f
                                                 Found module libsmime3.so with build-id: dec5e3e8ddfca7ab4eb5393320798237bc3dc1e8
                                                 Found module libnssutil3.so with build-id: bafb7c2f814be63ca73b4205bb12a292ebab8b7f
                                                 Found module libnss3.so with build-id: d19631624d51838d42878ca24703d63414a387eb
                                                 Found module libgio-2.0.so.0 with build-id: 3338d7168eab6039416e1b6dbb7ea3ef213cd129
                                                 Found module libxshmfence.so.1 with build-id: 8876d9ccf620858795724ca24b9e567585a77cec
                                                 Found module libglib-2.0.so.0 with build-id: 1fe1592d097fa28822c23e96d9f357583b48424d
                                                 Found module libgobject-2.0.so.0 with build-id: 97170184ce99115d663a8fe6587fd87fde6aa7e0
                                                 Found module librt.so.1 with build-id: 75484da2d6f1515189eefa076e0a40328834cd16
                                                 Found module libpthread.so.0 with build-id: 07c8f95b4f3251d08550217ad8a1f31066229996
                                                 Found module libdl.so.2 with build-id: 5abc547e7b0949f89f3c0e21ab0c8331a7440a8a
                                                 Stack trace of thread 65506:
                                                 #0  0x00007f192c1a1594 n/a (<pathTo>/VSCode-linux-x64/resources/app/node_modules.asar.unpacked/@parcel/watcher/build/Release/watcher.node + 0x47594)
                                                 #1  0x00007f192c1a1739 n/a (<pathTo>/VSCode-linux-x64/resources/app/node_modules.asar.unpacked/@parcel/watcher/build/Release/watcher.node + 0x47739)
                                                 #2  0x00007f192c1b44e1 n/a (<pathTo>/VSCode-linux-x64/resources/app/node_modules.asar.unpacked/@parcel/watcher/build/Release/watcher.node + 0x5a4e1)
                                                 #3  0x00007f192c1b2717 n/a (<pathTo>/VSCode-linux-x64/resources/app/node_modules.asar.unpacked/@parcel/watcher/build/Release/watcher.node + 0x58717)
                                                 #4  0x00007f192c19d67e n/a (<pathTo>/VSCode-linux-x64/resources/app/node_modules.asar.unpacked/@parcel/watcher/build/Release/watcher.node + 0x4367e)
                                                 #5  0x00007f1934eec4a7 __run_exit_handlers (libc.so.6 + 0x3f4a7)
                                                 #6  0x00007f1934eec64e exit (libc.so.6 + 0x3f64e)
                                                 #7  0x000055de2452c5f1 n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x74675f1)
                                                 #8  0x000055de2455923a n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x749423a)
                                                 #9  0x000055de2021c1f7 n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x31571f7)
                                                 #10 0x00003cb0000b2398 n/a (n/a + 0x0)
                                                 #11 0x00003cb000049541 n/a (n/a + 0x0)
                                                 #12 0x00003cb000049541 n/a (n/a + 0x0)
                                                 #13 0x00003cb000049b80 n/a (n/a + 0x0)
                                                 #14 0x00003cb0001d11b5 n/a (n/a + 0x0)
                                                 #15 0x00003cb0000475fb n/a (n/a + 0x0)
                                                 #16 0x00003cb000047383 n/a (n/a + 0x0)
                                                 #17 0x000055de202d5da9 n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x3210da9)
                                                 #18 0x000055de202d523a n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x321023a)
                                                 #19 0x000055de201d05ad n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x310b5ad)
                                                 #20 0x000055de2455882c n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x749382c)
                                                 #21 0x000055de1ec280fd n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x1b630fd)
                                                 #22 0x000055de1ec2b0cf n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x1b660cf)
                                                 #23 0x000055de1ed9aeb7 n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x1cd5eb7)
                                                 #24 0x000055de1ec3bc71 n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x1b76c71)
                                                 #25 0x00007f1934ed4b25 __libc_start_main (libc.so.6 + 0x27b25)
                                                 #26 0x000055de1e933daa n/a (<pathTo>/VSCode-linux-x64/code-insiders + 0x186edaa)

It looks like it may be happening with every "close" and I didn't notice it until now.
I see 4 crashes since the 30th when I upgraded(/deleted & replaced) from 1.58.
It happened again just now after I re-opened it, only to generate this report, then closed it, with what looks to be the very same kind of dump.

@bpasero bpasero added freeze-slow-crash-leak VS Code crashing, performance, freeze and memory leak issues file-watcher File watcher linux Issues with VS Code on Linux labels Nov 2, 2021
@bpasero
Copy link
Member

bpasero commented Nov 2, 2021

Can you please follow the steps in https://github.com/Microsoft/vscode/wiki/Native-Crash-Issues to get at more details around the crash and attach the result here? Thanks!

@bpasero bpasero added this to the November 2021 milestone Nov 2, 2021
@deepak1556 deepak1556 added the info-needed Issue requires more information from poster label Nov 2, 2021
@vindicatorr
Copy link
Author

Well, that's just odd. Using that command only created 2 files (client_id) and 1 folder (exthost Crash Reports).
In any case, I changed up the command a little more to:
$<pathTo>/VSCode-linux-x64/bin/code-insiders --crash-reporter-directory <pathTo>/crashReport --verbose --log trace

...
[342826:1102/094207.747913:INFO:CONSOLE(602)] "%cTRACE color: #888 SCMViewService#onDidAddRepository git:Git:file://<pathTo>/esp-idf/components/nghttp/nghttp2/third-party/neverbleed", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094207.749097:INFO:CONSOLE(602)] "%cTRACE color: #888 CommandService#executeCommand setContext", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094207.816020:INFO:CONSOLE(602)] "%cTRACE color: #888 CommandService#executeCommand setContext", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094207.817899:INFO:CONSOLE(602)] "%cTRACE color: #888 CommandService#executeCommand setContext", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094208.233037:INFO:CONSOLE(602)] "%cTRACE color: #888 MainThreadTimeline#emitChangeEvent: id=git-history, uri=undefined", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094215.266614:INFO:CONSOLE(29)] "%cTRACE color: #888 [code cache cleanup]: Starting to clean up old code cache folders.", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094215.267563:INFO:CONSOLE(29)] "%cTRACE color: #888 [storage cleanup]: Starting to clean up storage folders.", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094225.266482:INFO:CONSOLE(29)] "%cTRACE color: #888 [language pack cache cleanup]: Starting to clean up unused language packs.", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094250.891551:INFO:CONSOLE(602)] "%cTRACE color: #888 [text file model] resolve() - enter file://<pathTo>/esp-idf/components/driver/i2c.c", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094250.891613:INFO:CONSOLE(602)] "%cTRACE color: #888 [text file model] resolveFromFile() file://<pathTo>/esp-idf/components/driver/i2c.c", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[main 2021-11-02T14:42:51.122Z] menubarService#updateMenubar 1
[342826:1102/094251.153550:INFO:CONSOLE(602)] "%cTRACE color: #888 CommandService#executeCommand setContext", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094254.565856:INFO:CONSOLE(602)] "%cTRACE color: #888 CommandService#executeCommand notification.clear", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[main 2021-11-02T14:42:56.221Z] Lifecycle#window.on('close') - window ID 1
[main 2021-11-02T14:42:56.221Z] Lifecycle#unload() - window ID 1
[342826:1102/094256.222181:INFO:CONSOLE(602)] "%cTRACE color: #888 [lifecycle] onBeforeUnload (reason: 1)", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094256.227863:INFO:CONSOLE(18)] "%c[IPC Library: Pty Host] %c%cTRACE color: blue color:  color: #888 IPty#pid", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (18)
[342826:1102/094256.234722:INFO:CONSOLE(602)] "%cTRACE color: #888 [lifecycle] onBeforeUnload continues without veto", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[main 2021-11-02T14:42:56.235Z] Lifecycle#unload() - no veto (window ID 1)
[342826:1102/094256.236163:INFO:CONSOLE(602)] "%cTRACE color: #888 [lifecycle] onWillUnload (reason: 1)", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094256.236850:INFO:CONSOLE(602)] "%cTRACE color: #888 terminalInstance#dispose (instanceId: 1)", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094256.253438:INFO:CONSOLE(602)] "%cDEBUG background: #eee; color: #888 Terminal process exit (instanceId: 1) with code undefined", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094256.254078:INFO:CONSOLE(602)] "%cDEBUG background: #eee; color: #888 Terminal process exit (instanceId: 1) state 5", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094256.254751:INFO:CONSOLE(18)] "%c[IPC Library: Pty Host] %c%cTRACE color: blue color:  color: #888 ptyService#shutDown 1 false", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (18)
[342826:1102/094256.254835:INFO:CONSOLE(602)] "%cTRACE color: #888 terminalInstance#dispose (instanceId: 1)", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/workbench/workbench.desktop.main.js (602)
[342826:1102/094256.318089:INFO:CONSOLE(29)] "%cTRACE color: #888 profile not validated zsh zsh", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094256.324389:INFO:CONSOLE(29)] "%cTRACE color: #888 profile not validated fish fish", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094256.328885:INFO:CONSOLE(29)] "%cTRACE color: #888 profile not validated tmux tmux", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094256.333475:INFO:CONSOLE(29)] "%cTRACE color: #888 profile not validated pwsh pwsh", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (29)
[342826:1102/094256.506416:INFO:CONSOLE(18)] "%c[IPC Library: Pty Host] %c%cTRACE color: blue color:  color: #888 IPty#kill", source: vscode-file://vscode-app<pathTo>/VSCode-linux-x64/resources/app/out/vs/code/electron-browser/sharedProcess/sharedProcessMain.js (18)
[main 2021-11-02T14:42:56.520Z] menubarService#updateMenubar 1
[main 2021-11-02T14:42:56.548Z] StorageMainService: closed workspace storage (31ac783661662658dabadcf37b06eb6f)
[main 2021-11-02T14:42:56.555Z] Lifecycle#onBeforeCloseWindow.fire() - window ID 1
[main 2021-11-02T14:42:56.573Z] IPC Object URL: Removed channel vscode:d2493179-4741-42a7-b188-c375c76803b8.
[main 2021-11-02T14:42:56.573Z] Lifecycle#window.on('closed') - window ID 1
[main 2021-11-02T14:42:56.574Z] Lifecycle#onWillShutdown.fire()
[main 2021-11-02T14:42:56.575Z] IPC Object URL: Removed channel vscode:d787c390-c432-4842-a066-cca6a18dc4a2.
[main 2021-11-02T14:42:56.575Z] SharedProcess: disposing worker (reason: 'closed') {
  process: {
    moduleId: 'vs/platform/files/node/watcher/parcel/watcherApp',
    type: 'watcherServiceParcelSharedProcess'
  },
  reply: {
    windowId: 1,
    channel: 'vscode:createSharedProcessWorkerMessageChannelResult',
    nonce: 'e78af5fd-6969-4832-a051-4d1db75d58bb'
  }
}
[main 2021-11-02T14:42:56.583Z] Lifecycle#app.on(window-all-closed)
[main 2021-11-02T14:42:56.584Z] Lifecycle#app.on(before-quit)
[main 2021-11-02T14:42:56.584Z] Lifecycle#onBeforeShutdown.fire()
[main 2021-11-02T14:42:56.584Z] [WindowsStateHandler] onBeforeShutdown {
  lastActiveWindow: {
    workspaceIdentifier: undefined,
    folder: 'file://<pathTo>/<Project>',
    backupPath: '/home/username/.config/Code - Insiders/Backups/223af42d51f7b029a3cf9692e88100f7',
    remoteAuthority: undefined,
    uiState: [Object: null prototype] {
      mode: 0,
      x: 448,
      y: 185,
      width: 1024,
      height: 768
    }
  },
  lastPluginDevelopmentHostWindow: undefined,
  openedWindows: []
}
[main 2021-11-02T14:42:56.585Z] Lifecycle#app.on(will-quit)

Both times I ran the commands with 2 sets of parameters, they crashed on closing.

@deepak1556
Copy link
Collaborator

@bpasero watcher child process would be missing the crash reporter initialization unlike what extension host does

if (platform.isLinux) {
const crashReporterStartOptions: CrashReporterStartOptions = {
companyName: this._productService.crashReporter?.companyName || 'Microsoft',
productName: this._productService.crashReporter?.productName || this._productService.nameShort,
submitURL: '',
uploadToServer: false
};
const crashReporterId = this._environmentService.crashReporterId; // crashReporterId is set by the main process only when crash reporting is enabled by the user.
const appcenter = this._productService.appCenter;
const uploadCrashesToServer = !this._environmentService.crashReporterDirectory; // only upload unless --crash-reporter-directory is provided
if (uploadCrashesToServer && appcenter && crashReporterId && isUUID(crashReporterId)) {
const submitURL = appcenter[`linux-x64`];
crashReporterStartOptions.submitURL = submitURL.concat('&uid=', crashReporterId, '&iid=', crashReporterId, '&sid=', crashReporterId);
crashReporterStartOptions.uploadToServer = true;
}
// In the upload to server case, there is a bug in electron that creates client_id file in the current
// working directory. Setting the env BREAKPAD_DUMP_LOCATION will force electron to create the file in that location,
// For https://github.com/microsoft/vscode/issues/105743
const extHostCrashDirectory = this._environmentService.crashReporterDirectory || this._environmentService.userDataPath;
opts.env.BREAKPAD_DUMP_LOCATION = join(extHostCrashDirectory, `${ExtensionHostLogFileName} Crash Reports`);
opts.env.VSCODE_CRASH_REPORTER_START_OPTIONS = JSON.stringify(crashReporterStartOptions);
}
which gets utilized by bootstrap-fork eventually.

@bpasero
Copy link
Member

bpasero commented Nov 2, 2021

@deepak1556 hm, so we do use bootstrap-fork for spawning the workers:

this.child = fork(
this.environment.bootstrapPath,
[`--type=${this.configuration.process.type}`],
{ env: this.getEnv() }
);

But this means every process that wants to collect crash reports that uses bootstrap-fork needs that code from the extension host to work?

@deepak1556
Copy link
Collaborator

Yup that is correct, this only till we get to Electron 15 after which linux will start using crashpad and that code is not required anymore. So for now, we can just add this code to the watcher process launcher to debug this issue instead of a broader change ?

@bpasero
Copy link
Member

bpasero commented Nov 3, 2021

@vindicatorr I have a new insiders build ready that would produce a crash report (download).

Can you run from the command line: <path to download>/code-insiders <an existing folder to open> --crash-reporter-directory <some folder> and see if crash reports are produced for the file watcher?

@vindicatorr
Copy link
Author

Okay, so I tried looking at electron-minidump, but that just fails with ENOENT.
I just now figured out about breakpad and got that to work:
watcher_stackwalk.log
Let me know if that doesn't cut it, or I need to do something else.

@bpasero
Copy link
Member

bpasero commented Nov 3, 2021

@vindicatorr you should get a dmp file when you run with --crash-reporter-directory

@vindicatorr
Copy link
Author

Yes, and that was a stackwalk of that dmp file. Does that not contain the pertinent information?

@deepak1556
Copy link
Collaborator

Thanks for attempting to symbolicate the minidump but it is missing relevant symbol files, can you share the original dmp file ?

@vindicatorr
Copy link
Author

@deepak1556 deepak1556 added confirmed Issue has been confirmed by VS Code Team member and removed info-needed Issue requires more information from poster labels Nov 4, 2021
@deepak1556
Copy link
Collaborator

Symbolicated trace from the crashed thread:

Crash reason:  SIGSEGV /SEGV_MAPERR
Crash address: 0xffffc2bd1071640a
Process uptime: not available

Thread 0 (crashed)
 0  watcher.node!std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long) const [hashtable.h : 1816 + 0x4]
    rax = 0xffffc2bd1071640a   rdx = 0x00003d4044fbd380
    rcx = 0x55efc10abecf0909   rbx = 0x00003d4046150250
    rsi = 0x0000000000000000   rdi = 0x00007fe0371d4940
    rbp = 0x00007fff573c9c50   rsp = 0x00007fff573c9c20
     r8 = 0x3263c38c2bc47939    r9 = 0x00003d40450278b8
    r10 = 0x00000000c70f6907   r11 = 0x00007fe0371841e6
    r12 = 0x00007fe03e0be818   r13 = 0x00007fe0371d48c0
    r14 = 0x00007fe03e0c0ba8   r15 = 0x00003d4044fd6880
    rip = 0x00007fe037185196
    Found by: given as instruction pointer in context
 1  watcher.node!std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_erase(std::integral_constant<bool, true>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) [hashtable.h : 2182 + 0x18]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9c90
    rsp = 0x00007fff573c9c60   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037184240
    Found by: call frame info
 2  watcher.node!std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::erase(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) [hashtable.h : 921 + 0x13]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9cb0
    rsp = 0x00007fff573c9ca0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037183327
    Found by: call frame info
 3  watcher.node!std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::weak_ptr<DirTree>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<DirTree> > > >::erase(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) [unordered_map.h : 763 + 0x13]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9cd0
    rsp = 0x00007fff573c9cc0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037182adb
    Found by: call frame info
 4  watcher.node!DirTreeDeleter::operator()(DirTree*) [DirTree.cc : 9 + 0x12]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9d10
    rsp = 0x00007fff573c9ce0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037182816
    Found by: call frame info
 5  watcher.node!std::_Sp_counted_deleter<DirTree*, DirTreeDeleter, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() [shared_ptr_base.h : 442 + 0x13]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9d30
    rsp = 0x00007fff573c9d20   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371880b2
    Found by: call frame info
 6  watcher.node!std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() [shared_ptr_base.h : 168 + 0x17]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9da0
    rsp = 0x00007fff573c9d40   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03716a7e4
    Found by: call frame info
 7  watcher.node!std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() [shared_ptr_base.h : 705 + 0x8]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9dc0
    rsp = 0x00007fff573c9db0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037168931
    Found by: call frame info
 8  watcher.node!std::__shared_ptr<DirTree, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() [shared_ptr_base.h : 1154 + 0x10]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9de0
    rsp = 0x00007fff573c9dd0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03718289a
    Found by: call frame info
 9  watcher.node!std::shared_ptr<DirTree>::~shared_ptr() [shared_ptr.h : 122 + 0xc]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9e00
    rsp = 0x00007fff573c9df0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371828d4
    Found by: call frame info
10  watcher.node!InotifySubscription::~InotifySubscription() [InotifyBackend.hh : 10 + 0xc]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9e20
    rsp = 0x00007fff573c9e10   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a6106
    Found by: call frame info
11  watcher.node!void __gnu_cxx::new_allocator<InotifySubscription>::destroy<InotifySubscription>(InotifySubscription*) [new_allocator.h : 168 + 0xc]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9e40
    rsp = 0x00007fff573c9e30   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a612a
    Found by: call frame info
12  watcher.node!void std::allocator_traits<std::allocator<InotifySubscription> >::destroy<InotifySubscription>(std::allocator<InotifySubscription>&, InotifySubscription*) [alloc_traits.h : 531 + 0x13]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9e60
    rsp = 0x00007fff573c9e50   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a60d5
    Found by: call frame info
13  watcher.node!std::_Sp_counted_ptr_inplace<InotifySubscription, std::allocator<InotifySubscription>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() [shared_ptr_base.h : 528 + 0xb]
    rbx = 0x00003d4046150250   rbp = 0x00007fff573c9e90
    rsp = 0x00007fff573c9e70   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a5f9b
    Found by: call frame info
14  watcher.node!std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() [shared_ptr_base.h : 168 + 0x17]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9f00
    rsp = 0x00007fff573c9ea0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03716a7e4
    Found by: call frame info
15  watcher.node!std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() [shared_ptr_base.h : 705 + 0x8]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9f20
    rsp = 0x00007fff573c9f10   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037168931
    Found by: call frame info
16  watcher.node!std::__shared_ptr<InotifySubscription, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() [shared_ptr_base.h : 1154 + 0x10]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9f40
    rsp = 0x00007fff573c9f30   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a121e
    Found by: call frame info
17  watcher.node!std::shared_ptr<InotifySubscription>::~shared_ptr() [shared_ptr.h : 122 + 0xc]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9f60
    rsp = 0x00007fff573c9f50   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a123e
    Found by: call frame info
18  watcher.node!std::pair<int const, std::shared_ptr<InotifySubscription> >::~pair() [stl_pair.h : 211 + 0x10]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9f80
    rsp = 0x00007fff573c9f70   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a4714
    Found by: call frame info
19  watcher.node!void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false> >::destroy<std::pair<int const, std::shared_ptr<InotifySubscription> > >(std::pair<int const, std::shared_ptr<InotifySubscription> >*) [new_allocator.h : 168 + 0xc]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9fa0
    rsp = 0x00007fff573c9f90   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a4738
    Found by: call frame info
20  watcher.node!void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false> > >::destroy<std::pair<int const, std::shared_ptr<InotifySubscription> > >(std::allocator<std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false> >&, std::pair<int const, std::shared_ptr<InotifySubscription> >*) [alloc_traits.h : 531 + 0x13]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9fc0
    rsp = 0x00007fff573c9fb0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a3989
    Found by: call frame info
21  watcher.node!std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false>*) [hashtable_policy.h : 1891 + 0xb]
    rbx = 0x00003d4046151c08   rbp = 0x00007fff573c9ff0
    rsp = 0x00007fff573c9fd0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a2e05
    Found by: call frame info
22  watcher.node!std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<int const, std::shared_ptr<InotifySubscription> >, false>*) [hashtable_policy.h : 1913 + 0x13]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca020
    rsp = 0x00007fff573ca000   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a237b
    Found by: call frame info
23  watcher.node!std::_Hashtable<int, std::pair<int const, std::shared_ptr<InotifySubscription> >, std::allocator<std::pair<int const, std::shared_ptr<InotifySubscription> > >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, false> >::clear() [hashtable.h : 2299 + 0x1e]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca040
    rsp = 0x00007fff573ca030   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a1b9a
    Found by: call frame info
24  watcher.node!std::_Hashtable<int, std::pair<int const, std::shared_ptr<InotifySubscription> >, std::allocator<std::pair<int const, std::shared_ptr<InotifySubscription> > >, std::__detail::_Select1st, std::equal_to<int>, std::hash<int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, false> >::~_Hashtable() [hashtable.h : 1511 + 0xc]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca060
    rsp = 0x00007fff573ca050   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a16c8
    Found by: call frame info
25  watcher.node!std::unordered_multimap<int, std::shared_ptr<InotifySubscription>, std::hash<int>, std::equal_to<int>, std::allocator<std::pair<int const, std::shared_ptr<InotifySubscription> > > >::~unordered_multimap() [unordered_map.h : 1245 + 0xc]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca080
    rsp = 0x00007fff573ca070   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe0371a11fa
    Found by: call frame info
26  watcher.node!InotifyBackend::~InotifyBackend() [InotifyBackend.cc : 62 + 0x24]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca0a0
    rsp = 0x00007fff573ca090   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03719fd4a
    Found by: call frame info
27  watcher.node!void __gnu_cxx::new_allocator<InotifyBackend>::destroy<InotifyBackend>(InotifyBackend*) [new_allocator.h : 168 + 0x13]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca0c0
    rsp = 0x00007fff573ca0b0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037181095
    Found by: call frame info
28  watcher.node!void std::allocator_traits<std::allocator<InotifyBackend> >::destroy<InotifyBackend>(std::allocator<InotifyBackend>&, InotifyBackend*) [alloc_traits.h : 531 + 0x13]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca0e0
    rsp = 0x00007fff573ca0d0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037180fd5
    Found by: call frame info
29  watcher.node!std::_Sp_counted_ptr_inplace<InotifyBackend, std::allocator<InotifyBackend>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() [shared_ptr_base.h : 528 + 0xb]
    rbx = 0x00003d40450de1d0   rbp = 0x00007fff573ca110
    rsp = 0x00007fff573ca0f0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037180cb1
    Found by: call frame info
30  watcher.node!std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() [shared_ptr_base.h : 168 + 0x17]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca180
    rsp = 0x00007fff573ca120   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03716a7e4
    Found by: call frame info
31  watcher.node!std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() [shared_ptr_base.h : 705 + 0x8]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca1a0
    rsp = 0x00007fff573ca190   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037168931
    Found by: call frame info
32  watcher.node!std::__shared_ptr<Backend, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() [shared_ptr_base.h : 1154 + 0x10]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca1c0
    rsp = 0x00007fff573ca1b0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037166630
    Found by: call frame info
33  watcher.node!std::shared_ptr<Backend>::~shared_ptr() [shared_ptr.h : 122 + 0xc]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca1e0
    rsp = 0x00007fff573ca1d0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037166650
    Found by: call frame info
34  watcher.node!std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >::~pair() [stl_pair.h : 211 + 0x10]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca200
    rsp = 0x00007fff573ca1f0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717ecd6
    Found by: call frame info
35  watcher.node!void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true> >::destroy<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> > >(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >*) [new_allocator.h : 168 + 0xc]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca220
    rsp = 0x00007fff573ca210   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717ed06
    Found by: call frame info
36  watcher.node!void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true> > >::destroy<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> > >(std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true> >&, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >*) [alloc_traits.h : 531 + 0x13]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca240
    rsp = 0x00007fff573ca230   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717de45
    Found by: call frame info
37  watcher.node!std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true>*) [hashtable_policy.h : 1891 + 0xb]
    rbx = 0x00003d404521fa88   rbp = 0x00007fff573ca270
    rsp = 0x00007fff573ca250   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717cf95
    Found by: call frame info
38  watcher.node!std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, true>*) [hashtable_policy.h : 1913 + 0x13]
    rbx = 0x0000000000000000   rbp = 0x00007fff573ca2a0
    rsp = 0x00007fff573ca280   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717c5b9
    Found by: call frame info
39  watcher.node!std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::clear() [hashtable.h : 2299 + 0x1e]
    rbx = 0x0000000000000000   rbp = 0x00007fff573ca2c0
    rsp = 0x00007fff573ca2b0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717bcfa
    Found by: call frame info
40  watcher.node!std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::~_Hashtable() [hashtable.h : 1511 + 0xc]
    rbx = 0x0000000000000000   rbp = 0x00007fff573ca2e0
    rsp = 0x00007fff573ca2d0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03717b56c
    Found by: call frame info
41  watcher.node!std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::shared_ptr<Backend>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::shared_ptr<Backend> > > >::~unordered_map() [unordered_map.h : 102 + 0xc]
    rbx = 0x0000000000000000   rbp = 0x00007fff573ca300
    rsp = 0x00007fff573ca2f0   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe037180a5e
    Found by: call frame info
42  libc.so.6!__run_exit_handlers [exit.c : 113 + 0x7]
    rbx = 0x0000000000000000   rbp = 0x0000000000000000
    rsp = 0x00007fff573ca310   r12 = 0x00007fe03e0be818
    r13 = 0x00007fe0371d48c0   r14 = 0x00007fe03e0c0ba8
    r15 = 0x00003d4044fd6880   rip = 0x00007fe03deef4e5
    Found by: call frame info
43  libc.so.6!exit [exit.c : 143 + 0x5]
    rbx = 0x0000000000000000   rbp = 0x00007fff573ca390
    rsp = 0x00007fff573ca370   r12 = 0x000017fc00000000
    r13 = 0x00007fff573ca520   r14 = 0x0000000000000000
    r15 = 0x00003d4044fd8800   rip = 0x00007fe03deef660
    Found by: call frame info
44  code-oss!node::DefaultProcessExitHandler(node::Environment*, int) [environment.cc : 723 + 0x7]
    rbx = 0x0000000000000000   rbp = 0x00007fff573ca390
    rsp = 0x00007fff573ca380   r12 = 0x000017fc00000000
    r13 = 0x00007fff573ca520   r14 = 0x0000000000000000
    r15 = 0x00003d4044fd8800   rip = 0x0000556eafb42aa1
    Found by: call frame info
45  code-oss!node::Environment::Exit(int) [functional : 1888 + 0xe]
    rbx = 0x00003d40450e92a0   rbp = 0x00007fff573ca400
    rsp = 0x00007fff573ca3a0   r12 = 0x000017fc00000000
    r13 = 0x00007fff573ca520   r14 = 0x0000000000000000
    r15 = 0x00003d4044fd8800   rip = 0x0000556eafb6f6ea
    Found by: call frame info
46  code-oss!v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [api-arguments-inl.h : 158 + 0x9]
    rbx = 0x0000556eb0320500   rbp = 0x00007fff573ca570
    rsp = 0x00007fff573ca410   r12 = 0x000017fc00000000
    r13 = 0x00007fff573ca520   r14 = 0x000017fc08af2d81
    r15 = 0x0000000000000000   rip = 0x0000556eaabdc387
    Found by: call frame info
47  0x17fc000b2398
    rbx = 0x0000556eaabdbf20   rbp = 0x00007fff573ca590
    rsp = 0x00007fff573ca580   r12 = 0x0000000000000006
    r13 = 0x000017fc00000000   r14 = 0x0000000000000098
    r15 = 0x00007fff573ca5c8   rip = 0x000017fc000b2398
    Found by: call frame info
48  0x17fc00049541
    rbp = 0x00007fff573ca618   rsp = 0x00007fff573ca5a0
    rip = 0x000017fc00049541
    Found by: previous frame's frame pointer
49  0x17fc00049541
    rbp = 0x00007fff573ca688   rsp = 0x00007fff573ca628
    rip = 0x000017fc00049541
    Found by: previous frame's frame pointer
50  0x17fc00049541
    rbp = 0x00007fff573ca738   rsp = 0x00007fff573ca698
    rip = 0x000017fc00049541
    Found by: previous frame's frame pointer
51  0x17fc00049541
    rbp = 0x00007fff573ca7a8   rsp = 0x00007fff573ca748
    rip = 0x000017fc00049541
    Found by: previous frame's frame pointer
52  0x17fc000475fb
    rbp = 0x00007fff573ca7d8   rsp = 0x00007fff573ca7b8
    rip = 0x000017fc000475fb
    Found by: previous frame's frame pointer
53  0x17fc00047383
    rbp = 0x00007fff573ca840   rsp = 0x00007fff573ca7e8
    rip = 0x000017fc00047383
    Found by: previous frame's frame pointer
54  code-oss!v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [simulator.h : 144 + 0x8]
    rbp = 0x00007fff573ca990   rsp = 0x00007fff573ca850
    rip = 0x0000556eaac960a9
    Found by: previous frame's frame pointer
55  code-oss!v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [execution.cc : 466 + 0x8]
    rbx = 0x000017fc00000000   rbp = 0x00007fff573caa20
    rsp = 0x00007fff573ca9a0   r12 = 0x000017fc00000000
    r13 = 0x00003d404505aff0   r14 = 0x00007fff573cab98
    r15 = 0x0000000000000001   rip = 0x0000556eaac9553a
    Found by: call frame info
56  code-oss!v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [api.cc : 4968 + 0x5]
    rbx = 0x000017fc00000000   rbp = 0x00007fff573cab30
    rsp = 0x00007fff573caa30   r12 = 0x0000000000000006
    r13 = 0x00003d404505aff0   r14 = 0x00003d4045059028
    r15 = 0x00003d404510cd80   rip = 0x0000556eaab9072d
    Found by: call frame info
57  code-oss!node::Environment::RunTimers(uv_timer_s*) [env.cc : 793 + 0x14]
    rbx = 0x00003d4044fd8800   rbp = 0x00007fff573cac10
    rsp = 0x00007fff573cab40   r12 = 0x00003d404510c9e0
    r13 = 0x00003d404510cd80   r14 = 0xaaaaaaaaaaaaaaaa
    r15 = 0x00007fff573caba0   rip = 0x0000556eafb6ecdc
    Found by: call frame info
58  code-oss!uv__run_timers [timer.c : 177 + 0x5]
    rbx = 0x00003d4044fd8900   rbp = 0x00007fff573cac30
    rsp = 0x00007fff573cac20   r12 = 0x0000556ea8f605d0
    r13 = 0x0000556eb0373770   r14 = 0x0000556eb0373720
    r15 = 0x0000556eb0373720   rip = 0x0000556ea9cc832d
    Found by: call frame info
59  code-oss!uv_run [core.c : 376 + 0x8]
    rbx = 0x00007fff573cac50   rbp = 0x00007fff573caca0
    rsp = 0x00007fff573cac40   r12 = 0x0000556ea8f605d0
    r13 = 0x0000556eb0373770   r14 = 0x0000000000000000
    r15 = 0x0000556eb0373720   rip = 0x0000556ea9ccb2ff
    Found by: call frame info
60  code-oss!electron::NodeMain(int, char**) [node_main.cc : 253 + 0x7]
    rbx = 0x00003d4044fc50a0   rbp = 0x00007fff573caf90
    rsp = 0x00007fff573cacb0   r12 = 0x000017fc00000000
    r13 = 0x00003d4044fd8800   r14 = 0x00003d40450f4880
    r15 = 0x00003d4044fce7e0   rip = 0x0000556ea9e42c63
    Found by: call frame info
61  code-oss!main [electron_main.cc : 297 + 0xb]
    rbx = 0x00007fff573cafa0   rbp = 0x00007fff573cb090
    rsp = 0x00007fff573cafa0   r12 = 0x00007fff573cafa0
    r13 = 0x0000556ea9cdbd30   r14 = 0x00007fff573cb1b8
    r15 = 0x0000000000000003   rip = 0x0000556ea9cdbe91
    Found by: call frame info
62  libc.so.6!__libc_start_call_main [libc_start_call_main.h : 58 + 0x1a]
    rbx = 0x0000000000000000   rbp = 0x0000000000000003
    rsp = 0x00007fff573cb0a0   r12 = 0x00007fff573cb1b8
    r13 = 0x0000556ea9cdbd30   r14 = 0x0000000000000000
    r15 = 0x00007fe03fb6ec40   rip = 0x00007fe03ded3fd0
    Found by: call frame info
63  libc.so.6!__libc_start_main_alias_1 [libc-start.c : 409 + 0xd]
    rbx = 0x0000000000000000   rbp = 0x0000000000000003
    rsp = 0x00007fff573cb140   r12 = 0x00007fff573cb1b8
    r13 = 0x0000556ea9cdbd30   r14 = 0x0000000000000000
    r15 = 0x00007fe03fb6ec40   rip = 0x00007fe03ded407d
    Found by: call frame info
64  code-oss!_start + 0x2a
    rbx = 0x0000000000000000   rbp = 0x0000000000000000
    rsp = 0x00007fff573cb1a0   r12 = 0x0000556ea99d3940
    r13 = 0x00007fff573cb1b0   r14 = 0x0000000000000000
    r15 = 0x0000000000000000   rip = 0x0000556ea99d396a
    Found by: call frame info
65  0x7fff573cb1a8
    rbx = 0x0000000000000000   rbp = 0x0000000000000000
    rsp = 0x00007fff573cb1a8   r12 = 0x0000556ea99d3940
    r13 = 0x00007fff573cb1b0   r14 = 0x0000000000000000
    r15 = 0x0000000000000000   rip = 0x00007fff573cb1a8
    Found by: call frame info

Thread 1
 0  libc.so.6!epoll_wait [epoll_wait.c : 30 + 0x25]
    rax = 0xfffffffffffffffc   rdx = 0x0000000000000020
    rcx = 0x00007fe03dfced2e   rbx = 0x00003d4044fbbd00
    rsi = 0x00003d4044fbbc00   rdi = 0x000000000000001a
    rbp = 0x00007fe03c1dd4d0   rsp = 0x00007fe03c1dd470
     r8 = 0x0000000000000000    r9 = 0x0000000000000000
    r10 = 0x00000000ffffffff   r11 = 0x0000000000000293
    r12 = 0x00003d4044fd2100   r13 = 0x00003d4044feff00
    r14 = 0x00003d4044fbbc00   r15 = 0x00003d4044fd2100
    rip = 0x00007fe03dfced2e
    Found by: given as instruction pointer in context
 1  code-oss!epoll_dispatch [epoll.c : 198 + 0x8]
    rbx = 0x00003d4044fbbd00   rbp = 0x00007fe03c1dd4d0
    rsp = 0x00007fe03c1dd4a0   r12 = 0x00003d4044fd2100
    r13 = 0x00003d4044feff00   r14 = 0x00003d4044fbbc00
    r15 = 0x00003d4044fd2100   rip = 0x0000556eac733aab
    Found by: call frame info
 2  code-oss!event_base_loop [event.c : 512 + 0x14]
    rbx = 0x00003d4044fbbd00   rbp = 0x00007fe03c1dd570
    rsp = 0x00007fe03c1dd4e0   r12 = 0x00003d4044fd2718
    r13 = 0x0000000000000000   r14 = 0x00007fe03c1dd520
    r15 = 0x00003d4044fd2100   rip = 0x0000556eac7315cb
    Found by: call frame info
 3  code-oss!base::MessagePumpLibevent::Run(base::MessagePump::Delegate*) [message_pump_libevent.cc : 257 + 0xa]
    rbx = 0x00003d4044fbbd00   rbp = 0x00007fe03c1dd5f0
    rsp = 0x00007fe03c1dd580   r12 = 0x7fffffffffffffff
    r13 = 0x0000000000000000   r14 = 0x00003d4044fd3140
    r15 = 0x00003d4044fbbd88   rip = 0x0000556eac6121f7
    Found by: call frame info
 4  code-oss!base::sequence_manager::internal::ThreadControllerWithMessagePumpImpl::Run(bool, base::TimeDelta) [thread_controller_with_message_pump_impl.cc : 460 + 0x6]
    rbx = 0x00003d4044fbbd80   rbp = 0x00007fe03c1dd630
    rsp = 0x00007fe03c1dd600   r12 = 0x7fffffffffffffff
    r13 = 0x7fffffffffffffff   r14 = 0x0000000000000001
    r15 = 0x00003d4044fbbe20   rip = 0x0000556eac5c5d53
    Found by: call frame info
 5  code-oss!base::RunLoop::Run(base::Location const&) [run_loop.cc : 133 + 0x12]
    rbx = 0x00007fe03c1dd690   rbp = 0x00007fe03c1dd6d0
    rsp = 0x00007fe03c1dd640   r12 = 0x00007fe03c1dd6e0
    r13 = 0x00003d4044fe8ba8   r14 = 0x00007fe03c1dd740
    r15 = 0x00007fe03c1dd740   rip = 0x0000556eac598036
    Found by: call frame info
 6  code-oss!base::Thread::Run(base::RunLoop*) [thread.cc : 312 + 0x2a]
    rbx = 0x00007fe03c1dd740   rbp = 0x00007fe03c1dd710
    rsp = 0x00007fe03c1dd6e0   r12 = 0x00003d4044fe8bb8
    r13 = 0x00003d4044fe8ba8   r14 = 0x00007fe03c1dd6e0
    r15 = 0x00007fe03c1dd740   rip = 0x0000556eac5defc8
    Found by: call frame info
 7  code-oss!base::internal::ServiceThread::Run(base::RunLoop*) [service_thread.cc : 15 + 0x5]
    rbx = 0x00003d4044fce330   rbp = 0x00007fe03c1dd730
    rsp = 0x00007fe03c1dd720   r12 = 0x00003d4044fe8bb8
    r13 = 0x00003d4044fe8ba8   r14 = 0x00003d4044f877f0
    r15 = 0x00007fe03c1dd740   rip = 0x0000556eac5ced6d
    Found by: call frame info
 8  code-oss!base::Thread::ThreadMain() [thread.cc : 383 + 0xd]
    rbx = 0x00003d4044fce330   rbp = 0x00007fe03c1dd790
    rsp = 0x00007fe03c1dd740   r12 = 0x00003d4044fe8bb8
    r13 = 0x00003d4044fe8ba8   r14 = 0x00003d4044f877f0
    r15 = 0x00007fe03c1dd740   rip = 0x0000556eac5df161
    Found by: call frame info
 9  code-oss!base::(anonymous namespace)::ThreadFunc(void*) [platform_thread_posix.cc : 87 + 0xa]
    rbx = 0x00003d4044f878c0   rbp = 0x00007fe03c1dd7d0
    rsp = 0x00007fe03c1dd7a0   r12 = 0x00003d4044fe6d20
    r13 = 0x00003d4044fe8ba8   r14 = 0x00007fe03c1de640
    r15 = 0x00007fe03c1de640   rip = 0x0000556eac6014b6
    Found by: call frame info
10  libc.so.6!start_thread [pthread_create.c : 435 + 0x12]
    rbx = 0x00007fe03c1de640   rbp = 0x0000000000000000
    rsp = 0x00007fe03c1dd7e0   r12 = 0x00007fff573ca83e
    r13 = 0x00007fff573ca83f   r14 = 0x0000000000000000
    r15 = 0x00007fe03b9de000   rip = 0x00007fe03df3e927
    Found by: call frame info
11  libc.so.6!__GI___clone + 0x44
    rbx = 0x00007fe03c1de640   rbp = 0x0000000000000000
    rsp = 0x00007fe03c1dd880   r12 = 0x00007fff573ca83e
    r13 = 0x00007fff573ca83f   r14 = 0x0000000000000000
    r15 = 0x00007fe03b9de000   rip = 0x00007fe03dfce9e4
    Found by: call frame info

@deepak1556
Copy link
Collaborator

Steps to repro:

  1. Launch OSS on a directory ./scripts/code.sh <path-to-some-folder>
  2. Open a file from that directory
  3. Now close the application while the file is open
  4. CRASH

@deepak1556
Copy link
Collaborator

Based on the stack trace looks like INotifyBackend subscriptions are being accessed after the node process exit handlers have been invoked, @bpasero this reminds me of the CI test failure where we failed to unsubscribe from watcher events before exiting.

@bpasero
Copy link
Member

bpasero commented Nov 4, 2021

@deepak1556 with these steps (see video) I cannot reproduce a crash dump, are you doing anything different:

recording

@deepak1556
Copy link
Collaborator

Can you confirm from the process explorer if parcel watcher process is running ? Those steps looks solid.

@bpasero
Copy link
Member

bpasero commented Nov 4, 2021

Yeah it runs off the shared process:

image

(this is in Parallels, Linux ARM Ubuntu 20)

@deepak1556
Copy link
Collaborator

Latest insiders are from release/* branch, I don't think that has the crash reporter change. Can confirm crash with insiders using coredumpctl. Can you try with OSS ?

@bpasero
Copy link
Member

bpasero commented Nov 4, 2021

@deepak1556 I was specifically using the build I asked the OP to use and there is a folder being created for parcel watcher in the crash reporter dir.

@bpasero
Copy link
Member

bpasero commented Nov 4, 2021

@vindicatorr can you share how you found that crash, which command you used? Thanks.

@deepak1556
Copy link
Collaborator

Sorry missed that, let me try with that build.

@vindicatorr can correct me but it is tracked via systemd-coredump service, you can try

  1. sudo apt-get install systemd-coredump
  2. Run sysctl kernel.core_pattern, it should print something like kernel.core_pattern = |/lib/systemd/systemd-coredump
  3. Perform the steps for the crash
  4. Run coredumpctl which will list collected crashes
  5. Run coredumpctl info to get the last crash report stack trace

Screen Shot 2021-11-05 at 5 08 53 AM

@deepak1556
Copy link
Collaborator

let me try with that build.

Yup, I can get reports from the patched build.
recording

@vindicatorr
Copy link
Author

I missed the earlier message, but yeah, I didn't have to install systemd-coredump manually since it came with the systemd archlinux package.

For me (again, not having to do anything), my core_pattern is |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h.
Steps 4 and 5 are pretty much right, but you put in the "PID#" of the dump you want to view after info.

Mind you, I'd be expecting that you won't get a coredump if your crash-reporter didn't produce anything for you.
At first I was thinking it may have had to do with a lot of "includes" that the "watcher" would need to keep track of for my c esp-idf project.
There's also a warning about "...repository has 27 submodules which won't be opened automatically...". But I expect that's unrelated and a non-issue since it doesn't cause any problems and I expect is just a "heads-up". It other words, "watcher" isn't watching anything there since nothing was opened there.
But when I opened a very simple C# project, that too crashed, which dismisses my initial thought.

@bpasero
Copy link
Member

bpasero commented Nov 5, 2021

I can reproduce on Linux x64, but not ARM. Some facts:

@bpasero
Copy link
Member

bpasero commented Nov 6, 2021

@deepak1556 I am not able to reproduce this with bare node.js, so it must be Electron specific. I think you indicated this to me already back when we looked into the CI crash. Is it possible that Electron can fix this?

@bpasero bpasero added the electron Issues and items related to Electron label Nov 6, 2021
@bpasero
Copy link
Member

bpasero commented Nov 6, 2021

Looks like calling stop on shutdown might prevent this from happening, at least in my testing we seem to be having sufficient time on shutdown to deliver that message to the worker before it terminates: #136569

Anyway, we should probably leave this issue open to at least understand the difference in runtime behaviour between Electron and node.js.

bpasero added a commit that referenced this issue Nov 8, 2021
* watcher - stop service on Linux (#136264)

* watcher - cleanup

* only dispose in electron
@deepak1556
Copy link
Collaborator

deepak1556 commented Nov 8, 2021

@bpasero the difference wrt electron might be because without process reuse electron usually leaks node related handlers on process shutdown, hence I think the native module didn't get a chance for a proper cleanup based on node environment teardown. Once we update to Electron 15 were we will have process reuse by default, normal teardown of node environment is ensured https://github.com/electron/electron/blob/main/shell/renderer/electron_renderer_client.cc#L160

Do you want me to test with removing the above workaround as part of our Electron 15 branch ?

@bpasero
Copy link
Member

bpasero commented Nov 8, 2021

Ah yeah that makes sense, I think we are fine then once we move to E15.

deepak1556 added a commit that referenced this issue Nov 15, 2021
deepak1556 added a commit that referenced this issue Nov 15, 2021
deepak1556 added a commit that referenced this issue Nov 17, 2021
deepak1556 added a commit that referenced this issue Nov 18, 2021
deepak1556 added a commit that referenced this issue Nov 18, 2021
deepak1556 added a commit that referenced this issue Nov 19, 2021
@github-actions github-actions bot locked and limited conversation to collaborators Dec 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
confirmed Issue has been confirmed by VS Code Team member electron Issues and items related to Electron file-watcher File watcher freeze-slow-crash-leak VS Code crashing, performance, freeze and memory leak issues insiders-released Patch has been released in VS Code Insiders linux Issues with VS Code on Linux
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants
@bpasero @deepak1556 @vindicatorr and others