Troubleshooting 1.41.1 crash on armv6l

[deftdawg]

[Related to efforts to get CS running on Arm 32-bit #35]

• code-server version: 1.41.1 commit f51e045
• OS Version: Raspbian GNU/Linux 10 (buster) on Raspberry Pi Zero W / armv6l
• Node Version: 11.5.0 (last binaries distributed by nodejs.org that were built for armv6)

Description

Code Server will crash after serving most of the VS Code UI.

Code Server starts normally.
Upon taking a request it, it returns the application to the requesting user agent.
Code Server will continue to respond to requests provided those requests aren't allowed to run to completion (navigate away and back before UI has totally loaded, or close tab and open another tap).

If the UI is allowed to load to completion a few seconds after the UI appears to have fully loaded, Code Server exits with status code 0. At this point the browser receives a dialog
Cannot reconnect. Please reload the window. [Reload Window] [Cancel]

More info

Browser developer tools show everything loading normally until this file:
http://localhost:8080/static-f51e045cd5483561afc07694f39307fb673b6d1d/out/vs/workbench/services/extensions/worker/extensionHostWorker.js indicates it was cancelled with 0B having been received with a status of (cancelled) after 9.91 seconds.

Remote Agent log (all of them look like this)

cat .local/share/code-server/logs/2020*/remoteagent.log 
[2020-01-23 17:13:29.797] [remoteagent] [info] update#setState idle
[2020-01-23 17:13:56.640] [remoteagent] [info] request.url "/"
[2020-01-23 17:13:59.516] [remoteagent] [info] update#setState checking for updates
[2020-01-23 17:14:00.872] [remoteagent] [info] update#setState idle
[2020-01-23 17:14:57.893] [remoteagent] [error] Extension host stderr MODULE 5617: looking for "/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap-fork" in ["/home/pi/.node_modules","/home/pi/.node_libraries","/opt/nodejs/lib/node"]
MODULE 5617: load "/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap-fork.js" for module "."

[2020-01-23 17:14:58.060] [remoteagent] [error] Extension host stderr MODULE 5617: Module._load REQUEST nbin parent: .MODULE 5617: looking for "nbin" in ["/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/node_modules","/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/node_modules","/home/pi/node_modules","/home/node_modules","/node_modules","/home/pi/.node_modules","/home/pi/.node_libraries","/opt/nodejs/lib/node"]
MODULE 5617: Module._load REQUEST ./bootstrap parent: .
MODULE 5617: RELATIVE: requested: ./bootstrap set ID to: /home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap from .
MODULE 5617: looking for "/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap" in ["/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out"]
MODULE 5617: load "/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap.js" for module "/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap.js"

[2020-01-23 17:14:58.168] [remoteagent] [error] Extension host stderr MODULE 5617: Module._load REQUEST module parent: /home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap.js
MODULE 5617: load native module module
MODULE 5617: Module._load REQUEST path parent: /home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap.js
MODULE 5617: load native module path

[2020-01-23 17:14:58.237] [remoteagent] [error] Extension host stderr /opt/nodejs/bin/node[5617]: ../src/node_file.cc:892:void node::fs::InternalModuleStat(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args[0]->IsString()' failed.

[2020-01-23 17:14:58.249] [remoteagent] [error] Extension host ["%s %d: %s","TIMER",5617,"no 5000 list was found in insert, creating a new one"]
[2020-01-23 17:14:58.251] [remoteagent] [error] Extension host ["%s %d: %s","MODULE",5617,"Module._load REQUEST ./bootstrap-amd parent: ."]
[2020-01-23 17:14:58.251] [remoteagent] [error] Extension host ["%s %d: %s","MODULE",5617,"RELATIVE: requested: ./bootstrap-amd set ID to: /home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap-amd from ."]
[2020-01-23 17:14:58.251] [remoteagent] [error] Extension host ["%s %d: %s","MODULE",5617,"looking for \"/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/bootstrap-amd\" in [\"/home/pi/code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out\"]"]

Here is the console data with all NODE_DEBUG turned on:
node-console-debug.log

Steps to Reproduce

1. I run the following code to execute the code server under node:

if [ ! -e /usr/local/bin/node ]; then
  echo "Installing nodejs 11.5.0 (last build available for armv6)..."
  cat /proc/cpuinfo | grep -q "Pi Zero" && source <(curl -s https://raw.githubusercontent.com/sdesalas/node-pi-zero/master/install-node-v.last.sh)
fi

echo "Starting node with debug flags... "
NODE_DEBUG=stream,child_process,cluster,http,fs,tls,module,timer,net node code-serverdeftdawg-arm-debian-10-vsc1.41.1-linux-arm-built/out/vs/server/main.js --auth=none --disable-telemetry
echo "Exit Code was: $?"

2. Open a browser and navigate to code-server-ip:8080
3. Wait for page to load
4. Node will crash

If it would be helpful I make the build I'm working off available on my web server.

[deftdawg]

It's not the extensionHostWorker.js... I just re-tested in Firefox and that file loaded after just over 10 seconds on this test run.

I'm wondering if node is aborting because of this assertion from the remoteagent.log
/opt/nodejs/bin/node[6064]: ../src/node_file.cc:892:void node::fs::InternalModuleStat(const v8::FunctionCallbackInfo<v8::Value>&): Assertion `args[0]->IsString()' failed.

Firefox test run:

[kylecarbs]

Super helpful. Thanks for the report!

cc: @sr229

[sr229]

This is a wontfix.

We only support v12, and armv6l is EOL according to nodejs/nodejs.org#2205

[deftdawg]

Thanks for your help... the link you provided me lead me to the NodeJS Unofficial Builds site with builds of 12.14.1 for armv6.

Upgrading to 12.14.1 resolved the issue and allowed code-server to work armv6! It runs just fine now. 😄

I'm on to the last issue which is memory usage being way too high for ram constrained systems... specifically how the build script buffers the entire binary in memory before writing it to disk...

If I can replace that with a piped stream implementation(see solution #2), I think I should be able to build the standalone code server and package it on my 512MB Pi Zero 0 w.

[sr229]

Glad to know. If you manage to come up with a binary let me know how it turns out.

[sr229]

If possible, share us your steps so it can help us build code-server for armv7 @deftdawg, it'll be nice if we can build it for them since its been a blocker for us to build for 32-bit.