Support URL tunnelling (pass dwds UrlEncoder through to editors via daemon)

[DanTup]

@jonahwilliams @devoncarew this is a possible idea for supporting debugging on remote web environments.

The problem that this is trying to solve is that when we launch a web app through Flutter and want to use the Dart Debug extension, the app gets some injected JavaScript that includes the URL for the DWDS debug proxy service (which the Dart Debug extension will read and connect to). In a remote environment, the URL that is injected here is something like http://localhost:8123, but when this code executes on the end users machine, that URL is not accessible.

In order to make it accessible, we need to pass the URL back to the client (eg. VSO/GitPod) and ask it to set up a tunnel and translate the URL to one that will work for the end user. In the case of DWDS, this needs to be passed back in to Flutter so that Flutter can provide a handler to DWDS to map the URL as it's injecting the client.

My suggestion is to allow the Flutter daemon to send a request to the client (this is a request/response in the opposite direction to usual) to do this. The request would only be sent when a URL needs to be mapped (which generally means the client is specifically passing --no-web-browser-launch for web or --start-paused to the web-server device).

What do you think?

[jonahwilliams]

Seems similar to how the tool already sets up port-forwarding for connected devices, right? At least the client configuration part would need to be exposed in dwds I think though

[DanTup]

Seems similar to how the tool already sets up port-forwarding for connected devices, right?

The idea is similar, but for this the tunnelling/forwarding can't be done by Flutter, it needs to be exposed to the client to do (consider VS Online - the tunnelling involves setting up a proxy/tunnel into the VSO infrastructure, so we need to delegate it to them - which is done with an API call inside the VS Code extension).

At least the client configuration part would need to be exposed in dwds I think though

My understanding is that dart-lang/webdev#804 gives us the hook we need for dwds to call us back when it needs to translate the URL, so from there we can forward the request on to the client.

[DanTup]

@jonahwilliams @devoncarew I've pushed the implementation I'm currently using here for testing. It allows a client editor to pass a flag that enables this functionality, which will call back to the editor (using a request that goes in the opposite direction) when a URL needs mapping (this request is just passed through Flutter from dwds).

This is how URLs used within dwds (for example the URL of the debug service that the debug extension will use) are exposed by the infrastructure and the proxied URL passed back to dwds.

May be easier to review with whitespace ignored -> https://github.com/flutter/flutter/pull/44271/files?utf8=%E2%9C%93&diff=unified&w=1

(this will need tests, but I want to check this seems like a reasonable direction!)

[DanTup]

The build failure seems to be that I put the typedef for UrlTunneller in web_fs.dart and that's not allowed to be imported in run.dart. Is there a more appropriate place for this? @jonahwilliams

[jonahwilliams]

You can't expose anything from a build_runner folder through the tool normally, it needs to go through an indirection of some kind. The issue is that the build_runner deps don't exist in google3 or fuchsia-source, so there is a risk of the tool not building

[DanTup]

The only thing being imported is a simple typedef. I need it so I can inject one into context (see https://github.com/flutter/flutter/blob/3ee2f16793fab9a96768ab5f2c69032fbfb387ee/packages/flutter_tools/lib/src/commands/run.dart#L383-L385).

Is there somewhere suitable I can move that typedef that would allow me to remove this? (I guess it could pretty much be anywhere, I just couldn't find anything similar to be consistent with.

[jonahwilliams]

It doesn't matter what you import, because it will force all of the transitive imports to be resolved which will fail once it hits a build_runner/daemon import.

I would place it somewhere in base (net.dart?)

[DanTup]

@jonahwilliams @devoncarew Still looking for some feedback/review on this :-)

[jonahwilliams]

I'd like to avoid creating nested DI scopes if possible. Could we plumb this value directly through to the daemon instead?

[DanTup]

I pushed be95c6399b9c854000036f5cc374b7bacda2f628 - is this the sort of thing you had in mind? The condition is now handled in daemon.dart (it passes the urlTunneller function depending on the options).

[jonahwilliams]

Looks good to me, thank you!

[jonahwilliams]

For now, no - though we'll need to do something here in the future

[jonahwilliams]

The changes modulo the daemon API seem good to me. For that I would want @devoncarew to review

[DanTup]

Great, thanks! This still needs tests, but I'll wait to hear if @devoncarew is ok with the daemon API before I put tests around it.

[devoncarew]

@DanTup @jonahwilliams - the spec change here lgtm

[DanTup]

I pushed some tweaks:

• the spec said params for the response instead of result (result is what the daemon uses for it's outbound responses, and what JSON-RPC dictates)
• the implementation expects the result to be the string url directly, not an object with a field as-spec'd (I kept the spec and fixed the implementation, since having an object would allow us to extend this in future if we need more than just a url back)
• added a test that the calling exposeUrl does send the JSON request and handles the response

Let me know if this all still looks good.

[DanTup]

Rebased as there was a conflict (it was just a new import on the same line as one I added).

@jonahwilliams please lmk if you're happy with the tweaks mentioned in the comment above :-)

[jonahwilliams]

LGTM