Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: implement timers, JavaScript callbacks back into WebAssembly #25506

Closed
bradfitz opened this issue May 22, 2018 · 24 comments

Comments

Projects
None yet
6 participants
@bradfitz
Copy link
Member

commented May 22, 2018

After some discussion last week with @cherrymui, @rsc, @aclements, et al, we decided that users would be really sad if Go 1.11 was released with WebAssembly support and then realized they couldn't really use it in a browser well, without working timers & callbacks back into Go.

@cherrymui did a little prototype of callbacks and it was enough to show a demo of a Go webassembly program updating images in the DOM and responding to button clicks.

So we decided it was worth spending a bit of time and making sure the wasm support is what users will expect for Go 1.11.

@neelance, what's your availability look like to finish that in the next month or two?

I see you updated your branch (https://github.com/neelance/go/commits/wasm-wip) 3 days ago with neelance@773eb46

Do you need any help? @cherrymui and perhaps @aclements could help if needed.

Please use this issue number in any related CLs. Thanks!

@bradfitz bradfitz added this to the Go1.11 milestone May 22, 2018

neelance added a commit to neelance/go that referenced this issue May 23, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@gopherbot

This comment has been minimized.

Copy link

commented May 23, 2018

Change https://golang.org/cl/114197 mentions this issue: runtime, sycall/js: add support for callbacks from JavaScript

@neelance

This comment has been minimized.

Copy link
Member

commented May 23, 2018

Thanks for offering the help! Callbacks are already working quite well. I just cleaned up the commit some more and pushed the CL for initial feedback/review: https://go-review.googlesource.com/c/go/+/114197

Do we also want to use this to get support for http.Transport into this release? There is already a POC at https://github.com/johanbrandhorst/fetch.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented May 23, 2018

An http.DefaultTransport value implementing http.RoundTripper I assume? SGTM for Go 1.11. Let's do it.

If the author wants to send it via a PR or CL I can review. (PRs get converted into Gerrit these days)

@dmitshur

This comment has been minimized.

Copy link
Member

commented May 23, 2018

I'm happy to offer help with implementing and/or reviewing here as well.

I'm the author of the original Fetch API-based http.RoundTripper implementation in GopherJS that https://github.com/johanbrandhorst/fetch is based on, so I'm very familiar with how it works.

I've reached out to @johanbrandhorst and I'll coordinate next steps (regarding http.DefaultTransport) with him.

@johanbrandhorst

This comment has been minimized.

Copy link
Member

commented May 23, 2018

@shurcooL I'll try and get something in as soon as I can. I'm thinking we should implement the XHR transport as well and maybe make that the default as even among browsers that support WASM, they don't all support Fetch out of the box.

@neelance

This comment has been minimized.

Copy link
Member

commented May 23, 2018

fetch can be polyfilled and services like https://polyfill.io/ make it trivially easy to do so efficiently. I nowadays always prefer the newer (stable) APIs + polyfills.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented May 23, 2018

@neelance, in general I agree with that, but note that we want Go WebAssembly to work for users without the users needing to write or deal with any JavaScript. So unless that polyfill is part of some Go JS library, I'd rather we just fall back to an XHR RoundTripper if fetch is unavailable.

@neelance

This comment has been minimized.

Copy link
Member

commented May 23, 2018

@bradfitz This is a slippery slope. What about all the ES6 features that wasm_exec.js is using?

@neelance

This comment has been minimized.

Copy link
Member

commented May 23, 2018

For example fetch is supported by older browsers than async/await.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented May 23, 2018

@johanbrandhorst, @neelance, which browsers support WebAssembly & the ES6 features we're depending on but do NOT support Fetch?

If zero, then obviously we don't need XHR support.

https://caniuse.com/#feat=fetch suggests everything supports Fetch that already supports wasm.

@neelance

This comment has been minimized.

Copy link
Member

commented May 23, 2018

Oh yes, working on WebAssembly such much made me take it for granted. ;-) Yes, it looks like all browsers that support WebAssembly also support Fetch. No XHR needed.

@neelance

This comment has been minimized.

Copy link
Member

commented May 23, 2018

Same goes for async/await and the other ES6 features.

@dmitshur

This comment has been minimized.

Copy link
Member

commented May 23, 2018

As far as I know, all modern browsers support the basic Fetch API by now, but not all of them support streams, which are required for streaming the response body. E.g., see https://caniuse.com/#search=Streams. Firefox has javascript.options.streams and dom.streams.enabled flags that can enable it.

I think it should still be possible to use Fetch for those browsers, and avoid using Streams when they're not available.

neelance added a commit to neelance/go that referenced this issue May 24, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

@bradfitz bradfitz added the Arch-Wasm label May 24, 2018

@johanbrandhorst

This comment has been minimized.

Copy link
Member

commented May 24, 2018

Great, that sounds fine. I'll put together a PR as soon as I can (I want to try the PR workflow).

@gopherbot

This comment has been minimized.

Copy link

commented May 24, 2018

Change https://golang.org/cl/114515 mentions this issue: net/http: add js/wasm compatible DefaultTransport

gopherbot pushed a commit that referenced this issue May 30, 2018

net/http: add js/wasm compatible DefaultTransport
Adds a new Transport type for the js/wasm target that uses the
JavaScript Fetch API for sending HTTP requests. Support for
streaming response bodies is used when available, falling back
to reading the entire response into memory at once.

Updates #25506

Change-Id: Ie9ea433a1a2ed2f65b03c6cc84a16e70c06fcf5c
GitHub-Last-Rev: 6df6467
GitHub-Pull-Request: #25550
Reviewed-on: https://go-review.googlesource.com/114515
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@johanbrandhorst

This comment has been minimized.

Copy link
Member

commented May 30, 2018

With https://golang.org/cl/114515 merged, we didn't quite finish the discussion opened in https://go-review.googlesource.com/c/go/+/114515#129. I'd like to revisit and implement that discussion as the existing implementation may break some use cases. How can we identify if a test binary is running?

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented May 30, 2018

@johanbrandhorst, I can think of hacky ways to do it, but I'm not sure we have a supported way. For this application in particular I think we can look at os.Args[0] and see if it's "node".

@johanbrandhorst

This comment has been minimized.

Copy link
Member

commented May 30, 2018

That actually makes a lot of sense since Fetch isn't supported natively by Node anyway. I'll get a CL up.

@gopherbot

This comment has been minimized.

Copy link

commented May 31, 2018

Change https://golang.org/cl/115495 mentions this issue: net/http: use fake Transport network when running in Node

gopherbot pushed a commit that referenced this issue May 31, 2018

net/http: use fake Transport network when running in Node
Replaces the existing local loopback check with a check to see
whether the program is being interpreted by Node. This means
tests that are run with Node will use the fake network while still
allowing users who are using js/wasm to talk to local networks.

Updates #25506

Change-Id: I8bc3c6808fa29293b7ac5f77b186140c4ed90b51
GitHub-Last-Rev: 43d26af
GitHub-Pull-Request: #25663
Reviewed-on: https://go-review.googlesource.com/115495
Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

neelance added a commit to neelance/go that referenced this issue Jun 3, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@neelance

This comment has been minimized.

Copy link
Member

commented Jun 4, 2018

The net/http package does not compile any more:

src/net/http/roundtrip_js.go:23:10: undefined: t
src/net/http/roundtrip_js.go:200:11: undefined: errClosed
src/net/http/roundtrip_js.go:258:11: undefined: errClosed

Please fix.

neelance added a commit to neelance/go that referenced this issue Jun 4, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
@johanbrandhorst

This comment has been minimized.

Copy link
Member

commented Jun 4, 2018

My bad, I'll sort it.

johanbrandhorst added a commit to johanbrandhorst/go that referenced this issue Jun 4, 2018

net/http: Fix build errors on js/wasm target
The in-progress WASM port does not yet have sufficient automatic
testing performed against it, so these errors slipped through when
adding the new Fetch API backed Roundtripper.

Updates golang#25506

johanbrandhorst added a commit to johanbrandhorst/go that referenced this issue Jun 4, 2018

net/http: fix build errors on js/wasm target
The in-progress WASM port does not yet have sufficient automatic
testing performed against it, so these errors slipped through when
adding the new Fetch API backed http.Roundtripper.

Updates golang#25506
@gopherbot

This comment has been minimized.

Copy link

commented Jun 4, 2018

Change https://golang.org/cl/116076 mentions this issue: net/http: fix build errors on js/wasm target

johanbrandhorst added a commit to johanbrandhorst/go that referenced this issue Jun 4, 2018

net/http: fix build errors on js/wasm target
The in-progress WASM port does not yet have sufficient automatic
testing performed against it, so these errors slipped through when
adding the new Fetch API backed http.Roundtripper.

Updates golang#25506

gopherbot pushed a commit that referenced this issue Jun 4, 2018

net/http: fix build errors on js/wasm target
The in-progress WASM port does not yet have sufficient automatic
testing performed against it, so these errors slipped through when
adding the new Fetch API backed http.Roundtripper.

Updates #25506

Change-Id: I84c5832452e3e6067a02d926f67d01aaca66b837
GitHub-Last-Rev: 064062b
GitHub-Pull-Request: #25714
Reviewed-on: https://go-review.googlesource.com/116076
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

neelance added a commit to neelance/go that referenced this issue Jun 4, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

neelance added a commit to neelance/go that referenced this issue Jun 12, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

neelance added a commit to neelance/go that referenced this issue Jun 13, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

neelance added a commit to neelance/go that referenced this issue Jun 13, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

neelance added a commit to neelance/go that referenced this issue Jun 13, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

neelance added a commit to neelance/go that referenced this issue Jun 14, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates golang#18892
Updates golang#25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb

gopherbot pushed a commit that referenced this issue Jun 14, 2018

runtime, sycall/js: add support for callbacks from JavaScript
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.

JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.

Updates #18892
Updates #25506

Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
Reviewed-on: https://go-review.googlesource.com/114197
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@agnivade

This comment has been minimized.

Copy link
Member

commented Jun 18, 2018

@neelance - Just checking, is there anything else pending on this that you wanted to do ?

@neelance

This comment has been minimized.

Copy link
Member

commented Jun 18, 2018

I think we can close this issue. I only want to add some minor additions to the API for convenience (e.g. an easy way to create a promise).

@agnivade agnivade closed this Jun 18, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.