Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inspector: new API - Session.connectToMainThread #28870

Merged
merged 0 commits into from Sep 16, 2019

Conversation

@eugeneo
Copy link
Contributor

commented Jul 26, 2019

This API is designed to enable worker threads use Inspector protocol
on main thread (and other workers through NodeWorker domain).

Note that worker can cause dead lock by suspending itself. I will
work on a new API that will allow workers to be hidden from the
inspector.

Fixes: #28828

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
const callFrameId = (await pausedPromise).params.callFrames[0].callFrameId;

// Delay to ensure main thread is truly suspended
await new Promise((resolve) => setTimeout(resolve, 200));

This comment has been minimized.

Copy link
@ak239

ak239 Jul 26, 2019

Member

Why waiting for Debugger.paused is not enough in this case?

This comment has been minimized.

Copy link
@eugeneo

eugeneo Jul 26, 2019

Author Contributor

This makes sure main thread will wait while this worker is busy (e.g. if the API client decided to do IO)

test/parallel/test-inspector-connect-main-thread.js Outdated Show resolved Hide resolved

// This ensures that worker does not terminate while waiting for
// inspector response
function wrapCallbackToKeepThreadAlive(callback) {

This comment has been minimized.

Copy link
@ak239

ak239 Jul 26, 2019

Member

Do all clients of the new API will need something like this?

This comment has been minimized.

Copy link
@eugeneo

eugeneo Jul 26, 2019

Author Contributor

This will be fixed separately.

src/inspector_agent.h Outdated Show resolved Hide resolved
doc/api/errors.md Outdated Show resolved Hide resolved
doc/api/inspector.md Outdated Show resolved Hide resolved
doc/api/inspector.md Outdated Show resolved Hide resolved
a front-end connected to the Inspector WebSocket port.
Connects a session to the inspector back-end.

### session.connect()

This comment has been minimized.

Copy link
@addaleax

addaleax Jul 26, 2019

Member
Suggested change
### session.connect()
### session.connectToMainThread()

This comment has been minimized.

Copy link
@addaleax

addaleax Jul 26, 2019

Member

Also, to clarify: This connects to the parent thread and not necessarily the main thread, right?

This comment has been minimized.

Copy link
@eugeneo

eugeneo Jul 26, 2019

Author Contributor

Right! :)

This comment has been minimized.

Copy link
@eugeneo

eugeneo Jul 26, 2019

Author Contributor

Sorry, replied to a different thread.

Current implementation does not support "worker trees" - e.g. only workers with the main thread as a parent are reported by the NodeWorker domain. Is workers spawning workers intenteded use-case? Then it will need to be fixed and I expect that the fix will make this API work in a consistent way.

This comment has been minimized.

Copy link
@addaleax

addaleax Jul 26, 2019

Member

Is workers spawning workers intenteded use-case?

Yes, that’s possible, mostly because there’s no real reason to prevent it.

This comment has been minimized.

Copy link
@eugeneo

eugeneo Jul 26, 2019

Author Contributor

Then, I'll try fixing NodeWorker domain. This API will be consistent with what that domain reports.

This comment has been minimized.

Copy link
@eugeneo

eugeneo Jul 27, 2019

Author Contributor

This is the proposed fix: #28872

@eugeneo eugeneo added the inspector label Jul 26, 2019
doc/api/errors.md Outdated Show resolved Hide resolved
@eugeneo

This comment has been minimized.

Copy link
Contributor Author

commented Jul 28, 2019

I added a new commit so the callback is now executed in a micro task so there are less surprises.

@nodejs-github-bot

This comment has been minimized.

@Trott

This comment has been minimized.

Copy link
Member

commented Jul 29, 2019

The test added here failed in CI in the job that runs all tests in worker threads (using tools/test.py --worker).

@eugeneo

This comment has been minimized.

Copy link
Contributor Author

commented Jul 29, 2019

This test should only be started on a main thread, I marked it as such.

There are some other relevant test timeouts, I will be looking into them.

@nodejs-github-bot

This comment has been minimized.

Copy link

commented Jul 29, 2019

@Trott

This comment has been minimized.

Copy link
Member

commented Jul 29, 2019

CI passed. Is this ready for reviews? Or did you want to first look more closely at the timeouts you mentioned?

@Trott

This comment has been minimized.

Copy link
Member

commented Jul 31, 2019

@eugeneo CI passed. Did you still want to look more closely at the timeouts you mentioned?

@nodejs/inspector This could use some reviews.

Copy link
Member

left a comment

This LGTM and I'm +1 on the change and also don't have a good intuition about what we should do in the cases discussed (worker spawned worker connecting).

@eugeneo eugeneo force-pushed the eugeneo:connect-main-thread branch from 47f34a0 to 0fd75b8 Jul 31, 2019
@eugeneo

This comment has been minimized.

Copy link
Contributor Author

commented Jul 31, 2019

I rebased the change (there were some changes to how the Inspector works with the workers) and squashed individual changes

@ak239
ak239 approved these changes Jul 31, 2019
@nodejs-github-bot

This comment has been minimized.

@Trott

This comment has been minimized.

Copy link
Member

commented Aug 1, 2019

@Trott Trott added the author ready label Aug 1, 2019
@jasnell
jasnell approved these changes Aug 1, 2019
@Trott Trott added the semver-minor label Aug 1, 2019
a front-end connected to the Inspector WebSocket port.
Connects a session to the inspector back-end.

### session.connectToMainThread()

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 4, 2019

Member

To clarify again: This does connect to the parent thread and not the main thread, right? In that case the naming should reflect that, imo

This comment has been minimized.

Copy link
@eugeneo

eugeneo Aug 6, 2019

Author Contributor

All workers will connect to a main thread, I added a test case. This is based on #28872 that makes all workers be reported by the main thread.

const common = require('../common');

common.skipIfInspectorDisabled();
common.skipIfWorker();

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 4, 2019

Member

Why do we skip this if the process is a Worker thread? We should definitely also test what happens if this code is run inside a Worker, if we can. For now, this check breaks the test, because it doesn’t allow running this file in a Worker, not even from the test itself.

If there’s concern about the isMainThread check below, you can use a solution based on environment variables, e.g. the process.env.HAS_STARTED_WORKER bit in test/parallel/test-worker-exit-code.js. That’s a standard way of dealing with it in our test suite.

This comment has been minimized.

Copy link
@eugeneo

eugeneo Aug 6, 2019

Author Contributor

This test will not work in a worker as it expects the code to be ran on the main thread. I updated the check.

return new Promise((resolve) => session.once(notification, resolve));
}

function WaitForWorker() {

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 4, 2019

Member

We generally use lowerCamelCase for JS functions (ditto for the functions below)

This comment has been minimized.

Copy link
@eugeneo

eugeneo Aug 6, 2019

Author Contributor

Done.

function doConsoleLogIn5ms() {
setTimeout(() => {
console.log('Message for a test');
}, 5);

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 4, 2019

Member

Is this used somewhere?

This comment has been minimized.

Copy link
@eugeneo

eugeneo Aug 6, 2019

Author Contributor

Yes, it is for ensureListenerDoesNotInterrupt

if (isMainThread) {
RunTestWorker();
} else {
RunDebugging();

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 4, 2019

Member
Suggested change
RunDebugging();
runDebugging().then(common.mustCall());

This comment has been minimized.

Copy link
@eugeneo

eugeneo Aug 6, 2019

Author Contributor

Done

}

if (isMainThread) {
RunTestWorker();

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 4, 2019

Member
Suggested change
RunTestWorker();
runTestWorker().then(common.mustCall());

(Adding “Changes requested” because the test really does not work in its current form)

This comment has been minimized.

Copy link
@eugeneo

eugeneo Aug 6, 2019

Author Contributor

Done.

@addaleax addaleax removed the author ready label Aug 4, 2019
@eugeneo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 6, 2019

@addaleax thank you for the review. I did my best to update the test to better reflect semantics. Please, take another look.

Note that merge commit will go away after the rebase/squash, I did the merge so code review comments are easier to trace.

@eugeneo

This comment has been minimized.

Copy link
Contributor Author

commented Aug 16, 2019

@addaleax please let me know if there are still any changes that need to be implemented.

@Trott

This comment has been minimized.

Copy link
Member

commented Aug 25, 2019

@addaleax Does this look good to you now? Or are there still changes you'd like to see?

Copy link
Member

left a comment

Thanks for the ping; yeah, I think everything here has been addressed, I just didn’t have time to fully review this yet.

() => { consoleLogHappened = true; });
parentPort.postMessage({ consoleLogIn10ms: true });
while ((Date.now() - currentTime) < 50) {
// Spin wait for 200ms

This comment has been minimized.

Copy link
@addaleax

addaleax Aug 25, 2019

Member

The comment and the code seem to mismatch here 🙂

@addaleax addaleax dismissed their stale review Aug 25, 2019

done

@eugeneo eugeneo force-pushed the eugeneo:connect-main-thread branch from 6fe0497 to 9bb4b59 Aug 26, 2019
@Trott

This comment has been minimized.

Copy link
Member

commented Sep 16, 2019

This makes me think it should probably be in sequential:

  // Delay to ensure main thread is truly suspended
  await new Promise((resolve) => setTimeout(resolve, 50));

@nodejs/testing

@Trott

This comment has been minimized.

Copy link
Member

commented Sep 16, 2019

Not sure if this is an unrelated issue or not:

$ tools/test.py -j 96 --repeat 192 test/parallel/test-inspector-connect-main-thread.js 
=== release test-inspector-connect-main-thread ===                   
Path: parallel/test-inspector-connect-main-thread
Message NodeWorker.enable was sent
Skip child is done
Message Debugger.enable was sent
Message Runtime.enable was sent
Message Debugger.setBreakpointByUrl was sent
Message for a test
Message Debugger.evaluateOnCallFrame was sent
Message Debugger.resume was sent
Worker is done
out/Release/node[47372]: ../src/node_platform.cc:283:void node::PerIsolatePlatformData::Shutdown(): Assertion `(foreground_tasks_.Pop()) == nullptr' failed.
 1: 0x100078e0c node::Abort() [/Users/trott/io.js/out/Release/node]
 2: 0x100078bb9 node::AppendExceptionLine(node::Environment*, v8::Local<v8::Value>, v8::Local<v8::Message>, node::ErrorHandlingMode) [/Users/trott/io.js/out/Release/node]
 3: 0x1000cd679 node::PerIsolatePlatformData::Shutdown() [/Users/trott/io.js/out/Release/node]
 4: 0x1000cdd04 node::NodePlatform::UnregisterIsolate(v8::Isolate*) [/Users/trott/io.js/out/Release/node]
 5: 0x1000e69e5 node::worker::WorkerThreadData::~WorkerThreadData() [/Users/trott/io.js/out/Release/node]
 6: 0x1000e4e24 node::worker::Worker::Run() [/Users/trott/io.js/out/Release/node]
 7: 0x1000e6cd6 node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::$_2::__invoke(void*) [/Users/trott/io.js/out/Release/node]
 8: 0x7fff7c1002eb _pthread_body [/usr/lib/system/libsystem_pthread.dylib]
 9: 0x7fff7c103249 _pthread_start [/usr/lib/system/libsystem_pthread.dylib]
10: 0x7fff7c0ff40d thread_start [/usr/lib/system/libsystem_pthread.dylib]
Command: out/Release/node /Users/trott/io.js/test/parallel/test-inspector-connect-main-thread.js
--- CRASHED (Signal: 6) ---
@Trott

This comment has been minimized.

Copy link
Member

commented Sep 16, 2019

addaleax added a commit to addaleax/node that referenced this pull request Sep 17, 2019
Using `console.log()` likely interferes with the functionality of the
test, which also checks the interaction between inspector
and `console.log()` as part of the test. Using `process._rawDebug()`
solves that issue.

Refs: nodejs#28870
Refs: nodejs#29582
addaleax added a commit to addaleax/node that referenced this pull request Sep 17, 2019
While V8 itself should not have any remaining tasks on the queue
during platform shutdown, our inspector implementation may do so.
Thus, the checks verifying that no tasks are queued at that point
make some of the inspector tasks flaky.
Remove the checks and replace them by explicitly destroying all
tasks that are left.

Refs: nodejs#25653
Refs: nodejs#28870 (comment)
Trott added a commit to Trott/io.js that referenced this pull request Sep 17, 2019
Using `console.log()` likely interferes with the functionality of the
test, which also checks the interaction between inspector
and `console.log()` as part of the test. Using `process._rawDebug()`
solves that issue.

Refs: nodejs#28870
Refs: nodejs#29582

PR-URL: nodejs#29588
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
addaleax added a commit to addaleax/node that referenced this pull request Sep 17, 2019
While V8 itself should not have any remaining tasks on the queue
during platform shutdown, our inspector implementation may do so.
Thus, the checks verifying that no tasks are queued at that point
make some of the inspector tasks flaky.
Remove the checks and replace them by explicitly destroying all
tasks that are left.

Refs: nodejs#25653
Refs: nodejs#28870 (comment)
cjihrig added a commit to cjihrig/wasi that referenced this pull request Sep 18, 2019
Using `console.log()` likely interferes with the functionality of the
test, which also checks the interaction between inspector
and `console.log()` as part of the test. Using `process._rawDebug()`
solves that issue.

Refs: nodejs/node#28870
Refs: nodejs/node#29582

PR-URL: nodejs/node#29588
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
cjihrig added a commit to cjihrig/wasi that referenced this pull request Sep 18, 2019
Using `console.log()` likely interferes with the functionality of the
test, which also checks the interaction between inspector
and `console.log()` as part of the test. Using `process._rawDebug()`
solves that issue.

Refs: nodejs/node#28870
Refs: nodejs/node#29582

PR-URL: nodejs/node#29588
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
targos added a commit that referenced this pull request Sep 20, 2019
This API is designed to enable worker threads use Inspector protocol
on main thread (and other workers through NodeWorker domain).

Note that worker can cause dead lock by suspending itself. I will
work on a new API that will allow workers to be hidden from the
inspector.

Fixes: #28828
PR-URL: #28870
Reviewed-By: Aleksei Koziatinskii <ak239spb@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
targos added a commit that referenced this pull request Sep 20, 2019
Using `console.log()` likely interferes with the functionality of the
test, which also checks the interaction between inspector
and `console.log()` as part of the test. Using `process._rawDebug()`
solves that issue.

Refs: #28870
Refs: #29582

PR-URL: #29588
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Trott added a commit that referenced this pull request Sep 23, 2019
While V8 itself should not have any remaining tasks on the queue
during platform shutdown, our inspector implementation may do so.
Thus, the checks verifying that no tasks are queued at that point
make some of the inspector tasks flaky.
Remove the checks and replace them by explicitly destroying all
tasks that are left.

Refs: #25653
Refs: #28870 (comment)

PR-URL: #29587
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
targos added a commit that referenced this pull request Sep 23, 2019
While V8 itself should not have any remaining tasks on the queue
during platform shutdown, our inspector implementation may do so.
Thus, the checks verifying that no tasks are queued at that point
make some of the inspector tasks flaky.
Remove the checks and replace them by explicitly destroying all
tasks that are left.

Refs: #25653
Refs: #28870 (comment)

PR-URL: #29587
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
BridgeAR added a commit that referenced this pull request Sep 24, 2019
Notable changes:

* crypto:
  * Add `oaepLabel` option #29489
* deps:
  * Update V8 to 7.7.299.11 #28918
    * More efficient memory handling
    * Stack trace serialization got faster
    * The `Intl.NumberFormat` API gained new functionality
    * For more information: https://v8.dev/blog/v8-release-77
* events:
  * Add support for `EventTarget` in `once`
    #29498
* fs:
  * Expose memory file mapping flag `UV_FS_O_FILEMAP`
    #29260
* inspector:
  * New API - `Session.connectToMainThread`
    #28870
* process:
  * Initial SourceMap support via `env.NODE_V8_COVERAGE`
    #28960
* stream:
  * Make `_write()` optional when `_writev()` is implemented
    #29639
* tls:
  * Add option to override signature algorithms
    #29598
* util:
  * Add `encodeInto` to `TextEncoder`
    #29524
* worker:
  * The `worker_thread` module is now stable
    #29512
@BridgeAR BridgeAR referenced this pull request Sep 24, 2019
BridgeAR added a commit that referenced this pull request Sep 25, 2019
This API is designed to enable worker threads use Inspector protocol
on main thread (and other workers through NodeWorker domain).

Note that worker can cause dead lock by suspending itself. I will
work on a new API that will allow workers to be hidden from the
inspector.

Fixes: #28828
PR-URL: #28870
Reviewed-By: Aleksei Koziatinskii <ak239spb@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
BridgeAR added a commit that referenced this pull request Sep 25, 2019
Using `console.log()` likely interferes with the functionality of the
test, which also checks the interaction between inspector
and `console.log()` as part of the test. Using `process._rawDebug()`
solves that issue.

Refs: #28870
Refs: #29582

PR-URL: #29588
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
BridgeAR added a commit that referenced this pull request Sep 25, 2019
While V8 itself should not have any remaining tasks on the queue
during platform shutdown, our inspector implementation may do so.
Thus, the checks verifying that no tasks are queued at that point
make some of the inspector tasks flaky.
Remove the checks and replace them by explicitly destroying all
tasks that are left.

Refs: #25653
Refs: #28870 (comment)

PR-URL: #29587
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Eugene Ostroukhov <eostroukhov@google.com>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
BridgeAR added a commit that referenced this pull request Sep 25, 2019
Notable changes:

* crypto:
  * Add `oaepLabel` option #29489
* deps:
  * Update V8 to 7.7.299.11 #28918
    * More efficient memory handling
    * Stack trace serialization got faster
    * The `Intl.NumberFormat` API gained new functionality
    * For more information: https://v8.dev/blog/v8-release-77
* events:
  * Add support for `EventTarget` in `once`
    #29498
* fs:
  * Expose memory file mapping flag `UV_FS_O_FILEMAP`
    #29260
* inspector:
  * New API - `Session.connectToMainThread`
    #28870
* process:
  * Initial SourceMap support via `env.NODE_V8_COVERAGE`
    #28960
* stream:
  * Make `_write()` optional when `_writev()` is implemented
    #29639
* tls:
  * Add option to override signature algorithms
    #29598
* util:
  * Add `encodeInto` to `TextEncoder`
    #29524
* worker:
  * The `worker_thread` module is now stable
    #29512

PR-URL: #29695
BridgeAR added a commit that referenced this pull request Sep 25, 2019
Notable changes:

* crypto:
  * Add `oaepLabel` option #29489
* deps:
  * Update V8 to 7.7.299.11 #28918
    * More efficient memory handling
    * Stack trace serialization got faster
    * The `Intl.NumberFormat` API gained new functionality
    * For more information: https://v8.dev/blog/v8-release-77
* events:
  * Add support for `EventTarget` in `once`
    #29498
* fs:
  * Expose memory file mapping flag `UV_FS_O_FILEMAP`
    #29260
* inspector:
  * New API - `Session.connectToMainThread`
    #28870
* process:
  * Initial SourceMap support via `env.NODE_V8_COVERAGE`
    #28960
* stream:
  * Make `_write()` optional when `_writev()` is implemented
    #29639
* tls:
  * Add option to override signature algorithms
    #29598
* util:
  * Add `encodeInto` to `TextEncoder`
    #29524
* worker:
  * The `worker_thread` module is now stable
    #29512

PR-URL: #29695
BridgeAR added a commit that referenced this pull request Sep 25, 2019
Notable changes:

* crypto:
  * Add `oaepLabel` option #29489
* deps:
  * Update V8 to 7.7.299.11 #28918
    * More efficient memory handling
    * Stack trace serialization got faster
    * The `Intl.NumberFormat` API gained new functionality
    * For more information: https://v8.dev/blog/v8-release-77
* events:
  * Add support for `EventTarget` in `once`
    #29498
* fs:
  * Expose memory file mapping flag `UV_FS_O_FILEMAP`
    #29260
* inspector:
  * New API - `Session.connectToMainThread`
    #28870
* process:
  * Initial SourceMap support via `env.NODE_V8_COVERAGE`
    #28960
* stream:
  * Make `_write()` optional when `_writev()` is implemented
    #29639
* tls:
  * Add option to override signature algorithms
    #29598
* util:
  * Add `encodeInto` to `TextEncoder`
    #29524
* worker:
  * The `worker_thread` module is now stable
    #29512

PR-URL: #29695
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.