New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker debug #21364

Merged
merged 0 commits into from Sep 18, 2018

Conversation

@eugeneo
Contributor

eugeneo commented Jun 16, 2018

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • commit message follows commit guidelines

This is a proposed interface for debugging Node workers. It is largely similar to Chrome DevTools protocol with web-specific parts removed.

@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Jun 16, 2018

Contributor

This pull request includes commit from #21182

Contributor

eugeneo commented Jun 16, 2018

This pull request includes commit from #21182

@mscdex mscdex added the in progress label Jun 16, 2018

@eugeneo eugeneo changed the title from WIP: Worker debug to Worker debug Aug 14, 2018

@eugeneo eugeneo added C++ inspector worker and removed in progress labels Aug 14, 2018

@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Aug 14, 2018

Contributor

This PR is now ready for the review.

@nodejs/diagnostics @nodejs/v8-inspector @nodejs/workers

Contributor

eugeneo commented Aug 14, 2018

This PR is now ready for the review.

@nodejs/diagnostics @nodejs/v8-inspector @nodejs/workers

@ak239

ak239 approved these changes Aug 14, 2018 edited

I am wondering: should we instrument worker postMessage and onmessage with V8Inspector instrumentation for step-into worker and async stacks cross worker boundary?

Show outdated Hide outdated src/inspector/node_protocol_config.json
@jkrems

jkrems approved these changes Aug 14, 2018

Not the person to review the C++ specifics of this but overall looks good!

@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Aug 14, 2018

Contributor

I am wondering: should we instrument worker postMessage and onmessage with V8Inspector instrumentation for step-into worker and async stacks cross worker boundary?

Out of scope for this PR, IMHO.

Contributor

eugeneo commented Aug 14, 2018

I am wondering: should we instrument worker postMessage and onmessage with V8Inspector instrumentation for step-into worker and async stacks cross worker boundary?

Out of scope for this PR, IMHO.

@pavelfeldman

We've recently introduced flattened protocol operation for the Target domain for the sake of efficiency and I wonder if we should commit the new stuff into Node right away instead of landing the deprecated one... @dgozman, wdyt?

@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Aug 14, 2018

Contributor

@dgozman had approved this protocol file when the implementation had started.

Contributor

eugeneo commented Aug 14, 2018

@dgozman had approved this protocol file when the implementation had started.

@pavelfeldman

This comment has been minimized.

Show comment
Hide comment
@pavelfeldman

pavelfeldman Aug 14, 2018

Contributor

Well, since then we've introduced the flattened operation... Furthermore, we were considering moving the workers off the Target domain discovery, but that is still up in the air. Anyhow, let's see what he says.

Contributor

pavelfeldman commented Aug 14, 2018

Well, since then we've introduced the flattened operation... Furthermore, we were considering moving the workers off the Target domain discovery, but that is still up in the air. Anyhow, let's see what he says.

@eugeneo eugeneo added in progress and removed in progress labels Aug 17, 2018

@dgozman

Looks great! Thank you for going through this.

Show outdated Hide outdated src/inspector/node_protocol.pdl
Show outdated Hide outdated src/inspector/node_protocol.pdl
Show outdated Hide outdated src/inspector/node_protocol.pdl
@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Aug 21, 2018

Contributor

@dgozman @pavelfeldman qq: should worker inspectors have the NodeTarget domains?

Contributor

eugeneo commented Aug 21, 2018

@dgozman @pavelfeldman qq: should worker inspectors have the NodeTarget domains?

@dgozman

This comment has been minimized.

Show comment
Hide comment
@dgozman

dgozman Aug 21, 2018

@eugeneo If workers can spawn nested workers, it would be ideal to report them through the main NodeWorker domain if possible.

dgozman commented Aug 21, 2018

@eugeneo If workers can spawn nested workers, it would be ideal to report them through the main NodeWorker domain if possible.

@jasnell jasnell requested a review from addaleax Aug 21, 2018

@jasnell

This comment has been minimized.

Show comment
Hide comment

@jasnell jasnell added author ready and removed author ready labels Aug 21, 2018

Show outdated Hide outdated src/inspector/worker_agent.cc
Show outdated Hide outdated src/inspector/worker_agent.cc
Show outdated Hide outdated src/inspector/worker_inspector.h
Show outdated Hide outdated src/inspector_agent.cc
@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Aug 25, 2018

Contributor

@addaleax Thank you for the review. I addressed the comments and rebased the patch to make sure it works on top of current head.

Contributor

eugeneo commented Aug 25, 2018

@addaleax Thank you for the review. I addressed the comments and rebased the patch to make sure it works on top of current head.

@eugeneo eugeneo merged commit f28c6f7 into nodejs:master Sep 18, 2018

1 check was pending

continuous-integration/travis-ci/push The Travis CI build is in progress
Details
@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Sep 18, 2018

Contributor

Landed as f28c6f7

Contributor

eugeneo commented Sep 18, 2018

Landed as f28c6f7

@eugeneo eugeneo deleted the eugeneo:worker-debug branch Sep 18, 2018

@targos

This comment has been minimized.

Show comment
Hide comment
@targos

targos Sep 19, 2018

Member

Should this be backported to v10.x-staging? The change lands cleanly but the test fails:

$ out/Release/node --experimental-worker /home/mzasso/git/nodejs/v10.x/test/parallel/test-worker-debug.js
Test basic debug scenario
/home/mzasso/git/nodejs/v10.x/test/common/index.js:662
const crashOnUnhandledRejection = (err) => { throw err; };
                                             ^

AssertionError [ERR_ASSERTION]: Input A expected to strictly equal input B:
+ expected - actual

- 0
+ 2
    at WorkerSession.waitForBreakAfterCommand (/home/mzasso/git/nodejs/v10.x/test/parallel/test-worker-debug.js:111:12)
Member

targos commented Sep 19, 2018

Should this be backported to v10.x-staging? The change lands cleanly but the test fails:

$ out/Release/node --experimental-worker /home/mzasso/git/nodejs/v10.x/test/parallel/test-worker-debug.js
Test basic debug scenario
/home/mzasso/git/nodejs/v10.x/test/common/index.js:662
const crashOnUnhandledRejection = (err) => { throw err; };
                                             ^

AssertionError [ERR_ASSERTION]: Input A expected to strictly equal input B:
+ expected - actual

- 0
+ 2
    at WorkerSession.waitForBreakAfterCommand (/home/mzasso/git/nodejs/v10.x/test/parallel/test-worker-debug.js:111:12)

eugeneo added a commit to eugeneo/node that referenced this pull request Sep 19, 2018

inspector: workers debugging
Introduce a NodeTarget inspector domain modelled after ChromeDevTools
Target domain. It notifies inspector frontend attached to a main V8
isolate when workers are starting and allows passing messages to
inspectors on their isolates. All inspector functionality is enabled on
worker isolates.

PR-URL: nodejs#21364
Reviewed-By: Aleksei Koziatinskii <ak239spb@gmail.com>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Sep 19, 2018

Contributor

I definitely see a lot of value in introducing this in the same Node version the workers are introduced. This difference should be due to some V8 change - I adjusted the test case and the test works - https://github.com/eugeneo/node/tree/v10-workers-debug

Contributor

eugeneo commented Sep 19, 2018

I definitely see a lot of value in introducing this in the same Node version the workers are introduced. This difference should be due to some V8 change - I adjusted the test case and the test works - https://github.com/eugeneo/node/tree/v10-workers-debug

@eugeneo

This comment has been minimized.

Show comment
Hide comment
@eugeneo

eugeneo Sep 19, 2018

Contributor

@targos - sorry, forgot to tag you. I also did a CI run - seems ok - https://ci.nodejs.org/job/node-test-commit/21627/

Contributor

eugeneo commented Sep 19, 2018

@targos - sorry, forgot to tag you. I also did a CI run - seems ok - https://ci.nodejs.org/job/node-test-commit/21627/

@targos

This comment has been minimized.

Show comment
Hide comment
@targos

targos Sep 19, 2018

Member

@eugeneo Excellent! Would you like to open a backport PR?
It looks like you could just create one from that branch and target v10.x-staging. Feel free to ping me on it and link the CI run.

Member

targos commented Sep 19, 2018

@eugeneo Excellent! Would you like to open a backport PR?
It looks like you could just create one from that branch and target v10.x-staging. Feel free to ping me on it and link the CI run.

@eugeneo eugeneo referenced this pull request Sep 19, 2018

Closed

[v10.x backport] inspector: workers debugging #22954

2 of 2 tasks complete

@targos targos added this to Backport requested in v10.x Sep 23, 2018

targos added a commit that referenced this pull request Sep 25, 2018

inspector: workers debugging
Introduce a NodeTarget inspector domain modelled after ChromeDevTools
Target domain. It notifies inspector frontend attached to a main V8
isolate when workers are starting and allows passing messages to
inspectors on their isolates. All inspector functionality is enabled on
worker isolates.

Backport-PR-URL: #22954
PR-URL: #21364
Reviewed-By: Aleksei Koziatinskii <ak239spb@gmail.com>
Reviewed-By: Jan Krems <jan.krems@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>

@targos targos moved this from Backport requested to Backported in v10.x Sep 25, 2018

@targos targos referenced this pull request Oct 7, 2018

Merged

Release proposal: v10.12.0 #23313

targos added a commit that referenced this pull request Oct 10, 2018

2018-10-10, Version 10.12.0 (Current)
Notable changes:

* assert
  * The diff output is now a tiny bit improved by sorting object
    properties when inspecting the values that are compared with each
    other. #22788
* cli
  * The options parser now normalizes `_` to `-` in all multi-word
    command-line flags, e.g. `--no_warnings` has the same effect as
    `--no-warnings`. #23020
  * Added bash completion for the `node` binary. To generate a bash
    completion script, run `node --completion-bash`. The output can be
    saved to a file which can be sourced to enable completion.
    #20713
* crypto
  * Added support for PEM-level encryption.
    #23151
  * Added an API asymmetric key pair generation. The new methods
    `crypto.generateKeyPair` and `crypto.generateKeyPairSync` can be
    used to generate public and private key pairs. The API supports
    RSA, DSA and EC and a variety of key encodings (both PEM and DER).
    #22660
* fs
  * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If
    this option is set to true, non-existing parent folders will be
    automatically created. #21875
* http2
  * Added a `'ping'` event to `Http2Session` that is emitted whenever a
    non-ack `PING` is received.
    #23009
  * Added support for the `ORIGIN` frame.
    #22956
  * Updated nghttp2 to 1.34.0. This adds RFC 8441 extended connect
    protocol support to allow use of WebSockets over HTTP/2.
    #23284
* module
  * Added `module.createRequireFromPath(filename)`. This new method can
    be used to create a custom require function that will resolve
    modules relative to the filename path.
    #19360
* process
  * Added a `'multipleResolves'` process event that is emitted whenever
    a `Promise` is attempted to be resolved multiple times, e.g. if the
    `resolve` and `reject` functions are both called in a `Promise`
    executor. #22218
* url
  * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These
    methods can be used to correctly convert between file: URLs and
    absolute paths. #22506
* util
  * Added the `sorted` option to `util.inspect()`. If set to `true`,
    all properties of an object and Set and Map entries will be sorted
    in the returned string. If set to a function, it is used as a
    compare function. #22788
  * The `util.instpect.custom` symbol is now defined in the global
    symbol registry as `Symbol.for('nodejs.util.inspect.custom')`.
    #20857
  * Added support for `BigInt` numbers in `util.format()`.
    #22097
* V8 API
  * A number of V8 C++ APIs have been marked as deprecated since they
    have been removed in the upstream repository. Replacement APIs
    are added where necessary. #23159
* Windows
  * The Windows msi installer now provides an option to automatically
    install the tools required to build native modules.
    #22645
* Workers
  * Debugging support for Workers using the DevTools protocol has been
    implemented. #21364
  * The public `inspector` module is now enabled in Workers.
    #22769
* Added new collaborators:
  * digitalinfinity - Hitesh Kanwathirtha

PR-URL: #23313

targos added a commit that referenced this pull request Oct 10, 2018

2018-10-10, Version 10.12.0 (Current)
Notable changes:

* assert
  * The diff output is now a tiny bit improved by sorting object
    properties when inspecting the values that are compared with each
    other. #22788
* cli
  * The options parser now normalizes `_` to `-` in all multi-word
    command-line flags, e.g. `--no_warnings` has the same effect as
    `--no-warnings`. #23020
  * Added bash completion for the `node` binary. To generate a bash
    completion script, run `node --completion-bash`. The output can be
    saved to a file which can be sourced to enable completion.
    #20713
* crypto
  * Added support for PEM-level encryption.
    #23151
  * Added an API asymmetric key pair generation. The new methods
    `crypto.generateKeyPair` and `crypto.generateKeyPairSync` can be
    used to generate public and private key pairs. The API supports
    RSA, DSA and EC and a variety of key encodings (both PEM and DER).
    #22660
* fs
  * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If
    this option is set to true, non-existing parent folders will be
    automatically created. #21875
* http2
  * Added a `'ping'` event to `Http2Session` that is emitted whenever a
    non-ack `PING` is received.
    #23009
  * Added support for the `ORIGIN` frame.
    #22956
  * Updated nghttp2 to 1.34.0. This adds RFC 8441 extended connect
    protocol support to allow use of WebSockets over HTTP/2.
    #23284
* module
  * Added `module.createRequireFromPath(filename)`. This new method can
    be used to create a custom require function that will resolve
    modules relative to the filename path.
    #19360
* process
  * Added a `'multipleResolves'` process event that is emitted whenever
    a `Promise` is attempted to be resolved multiple times, e.g. if the
    `resolve` and `reject` functions are both called in a `Promise`
    executor. #22218
* url
  * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These
    methods can be used to correctly convert between file: URLs and
    absolute paths. #22506
* util
  * Added the `sorted` option to `util.inspect()`. If set to `true`,
    all properties of an object and Set and Map entries will be sorted
    in the returned string. If set to a function, it is used as a
    compare function. #22788
  * The `util.instpect.custom` symbol is now defined in the global
    symbol registry as `Symbol.for('nodejs.util.inspect.custom')`.
    #20857
  * Added support for `BigInt` numbers in `util.format()`.
    #22097
* V8 API
  * A number of V8 C++ APIs have been marked as deprecated since they
    have been removed in the upstream repository. Replacement APIs
    are added where necessary. #23159
* Windows
  * The Windows msi installer now provides an option to automatically
    install the tools required to build native modules.
    #22645
* Workers
  * Debugging support for Workers using the DevTools protocol has been
    implemented. #21364
  * The public `inspector` module is now enabled in Workers.
    #22769
* Added new collaborators:
  * digitalinfinity - Hitesh Kanwathirtha

PR-URL: #23313

targos added a commit that referenced this pull request Oct 10, 2018

2018-10-10, Version 10.12.0 (Current)
Notable changes:

* assert
  * The diff output is now a tiny bit improved by sorting object
    properties when inspecting the values that are compared with each
    other. #22788
* cli
  * The options parser now normalizes `_` to `-` in all multi-word
    command-line flags, e.g. `--no_warnings` has the same effect as
    `--no-warnings`. #23020
  * Added bash completion for the `node` binary. To generate a bash
    completion script, run `node --completion-bash`. The output can be
    saved to a file which can be sourced to enable completion.
    #20713
* crypto
  * Added support for PEM-level encryption.
    #23151
  * Added an API asymmetric key pair generation. The new methods
    `crypto.generateKeyPair` and `crypto.generateKeyPairSync` can be
    used to generate public and private key pairs. The API supports
    RSA, DSA and EC and a variety of key encodings (both PEM and DER).
    #22660
* fs
  * Added a `recursive` option to `fs.mkdir` and `fs.mkdirSync`. If
    this option is set to true, non-existing parent folders will be
    automatically created. #21875
* http2
  * Added a `'ping'` event to `Http2Session` that is emitted whenever a
    non-ack `PING` is received.
    #23009
  * Added support for the `ORIGIN` frame.
    #22956
  * Updated nghttp2 to 1.34.0. This adds RFC 8441 extended connect
    protocol support to allow use of WebSockets over HTTP/2.
    #23284
* module
  * Added `module.createRequireFromPath(filename)`. This new method can
    be used to create a custom require function that will resolve
    modules relative to the filename path.
    #19360
* process
  * Added a `'multipleResolves'` process event that is emitted whenever
    a `Promise` is attempted to be resolved multiple times, e.g. if the
    `resolve` and `reject` functions are both called in a `Promise`
    executor. #22218
* url
  * Added `url.fileURLToPath(url)` and `url.pathToFileURL(path)`. These
    methods can be used to correctly convert between file: URLs and
    absolute paths. #22506
* util
  * Added the `sorted` option to `util.inspect()`. If set to `true`,
    all properties of an object and Set and Map entries will be sorted
    in the returned string. If set to a function, it is used as a
    compare function. #22788
  * The `util.instpect.custom` symbol is now defined in the global
    symbol registry as `Symbol.for('nodejs.util.inspect.custom')`.
    #20857
  * Added support for `BigInt` numbers in `util.format()`.
    #22097
* V8 API
  * A number of V8 C++ APIs have been marked as deprecated since they
    have been removed in the upstream repository. Replacement APIs
    are added where necessary. #23159
* Windows
  * The Windows msi installer now provides an option to automatically
    install the tools required to build native modules.
    #22645
* Workers
  * Debugging support for Workers using the DevTools protocol has been
    implemented. #21364
  * The public `inspector` module is now enabled in Workers.
    #22769
* Added new collaborators:
  * digitalinfinity - Hitesh Kanwathirtha

PR-URL: #23313
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment