Symbolication of local builds doesn't work on dbaron's machine

[mstange]

Continued from bug 1352893.

Here's the log:

[starting] reading existing symbol dump for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 from the objdir
attempting to read sym file for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 at /home/dbaron/crashreporter-symbols/libxul.so/855015EE8D563E4B2E00CB39C4A950790/libxul.so.sym
[failure] reading existing symbol dump for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 from the objdir
(new Error("", (void 0)))
fetching symbol dump for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 from http://symbols.mozilla.org/libxul.so/855015EE8D563E4B2E00CB39C4A950790/libxul.so.sym
[starting] requesting symbols for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 from the symbol server
fetching symbol dump for library   from http://symbols.mozilla.org///.sym
[starting] requesting symbols for library   from the symbol server
[failure] requesting symbols for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 from the symbol server
(new Error("got error status 404", "resource://gre/modules/commonjs/toolkit/loader.js -> resource://jid0-edalmuivkozlouyij0lpdx548bc-at-jetpack/lib/symbol-store.js", 95))
[starting] dumping symbols for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 located at /home/dbaron/bin/running-firefox/libxul.so using nm

@dbaron, it's worrying that "[starting] dumping symbols for library libxul.so" is the last message about libxul.so. Either nm is taking a very long time, or we're failing somewhere and not reporting the failure. Did the status overlay at the top of perf.html that says "Waiting for symbol tables for library XUL..." ever go away?

[dbaron]

@mstange I don't think it did, but I'm not sure how long I waited. How long should I wait for it?

[mstange]

Not more than two minutes. Can you also check your process monitor and see whether nm shows up?

[dbaron]

So an nm process: /usr/bin/nm --demangle /home/dbaron/bin/running-firefox/libxul.so does show up for a bit -- somewhere between 10 and 30 seconds -- but when it completes, I still don't have symbols for libxul.so, and the UI in perf.html still says "Waiting for symbol tables for libraries libxul.so, libnspr4.so, libXi.so.6, libplc4.so, libnss3.so, im-fcitx.s..."

I don't see anything in the output mentioning libxul.so after:

[starting] dumping symbols for library libxul.so 855015EE8D563E4B2E00CB39C4A950790 located at /home/dbaron/bin/running-firefox/libxul.so using nm

[mstange]

That means that something goes wrong in or after this function: https://github.com/devtools-html/Gecko-Profiler-Addon/blob/master/lib/nm-syms.js#L101

Could you build the add-on yourself by following the steps at https://github.com/devtools-html/Gecko-Profiler-Addon/#development , add some console.log()s, and see how far we get?

If you don't have time for this, I can prepare an xpi with more debug information for you, but doing it that way might involve a lot of back-and-forth until we've narrowed down the issue.

[dbaron]

When I follow the (non-Mac) instructions, I get:

~/builds/Gecko-Profiler-Addon $ npm run -- --binary /home/dbaron/builds/yarra/firefox-debugopt/firefox
npm ERR! Linux 4.8.0-46-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "--" "--binary" "/home/dbaron/builds/yarra/firefox-debugopt/firefox"
npm ERR! node v4.2.6
npm ERR! npm  v3.5.2

npm ERR! missing script: --binary
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /home/dbaron/builds/Gecko-Profiler-Addon/npm-debug.log

and the npm-debug.log says:

0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/nodejs',
1 verbose cli   '/usr/bin/npm',
1 verbose cli   'run',
1 verbose cli   '--',
1 verbose cli   '--binary',
1 verbose cli   '/home/dbaron/builds/yarra/firefox-debugopt/firefox' ]
2 info using npm@3.5.2
3 info using node@v4.2.6
4 verbose stack Error: missing script: --binary
4 verbose stack     at run (/usr/share/npm/lib/run-script.js:147:19)
4 verbose stack     at /usr/share/npm/lib/run-script.js:57:5
4 verbose stack     at /usr/share/npm/node_modules/read-package-json/read-json.js:345:5
4 verbose stack     at checkBinReferences_ (/usr/share/npm/node_modules/read-package-json/read-json.js:309:45)
4 verbose stack     at final (/usr/share/npm/node_modules/read-package-json/read-json.js:343:3)
4 verbose stack     at then (/usr/share/npm/node_modules/read-package-json/read-json.js:113:5)
4 verbose stack     at /usr/share/npm/node_modules/read-package-json/read-json.js:300:12
4 verbose stack     at /usr/share/npm/node_modules/graceful-fs/graceful-fs.js:76:16
4 verbose stack     at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)
5 verbose cwd /home/dbaron/builds/Gecko-Profiler-Addon
6 error Linux 4.8.0-46-generic
7 error argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "--" "--binary" "/home/dbaron/builds/yarra/firefox-debugopt/firefox"
8 error node v4.2.6
9 error npm  v3.5.2
10 error missing script: --binary
11 error If you need help, you may report this error at:
11 error     <https://github.com/npm/npm/issues>
12 verbose exit [ 1, true ]

[mstange]

Argh, those instructions are wrong. It should be npm start -- --binary /path/to/firefox, not npm run. (npm start then calls jpm run, which was apparently the source of my confusion.) Sorry!

[dbaron]

OK, I poked around a little bit: as far as I can tell, feedProcessStdoutInto never gets either a close or an error event. (But there are tons of valid processLine calls.)

[mstange]

Interesting, thanks for looking!

So maybe this is a problem with the child_process API. That's an SDK API that's going to be removed soon anyway. Switching to Subprocess.jsm might fix this.
And it looks like @squarewave already has done the work to switch us to Subprocess.jsm in squarewave/gecko-profiler-webextension@5364e3a .

[clarkbw]

With the new WebExtension out now, perhaps @dbaron can try again with a Nightly build?

[dbaron]

Yes, this works now. Thanks.