Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add net module to utility process #40017

Merged
merged 30 commits into from Jan 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
3607ae6
chore: initial prototype of net api from utility process
deepak1556 Sep 19, 2023
701c227
chore: update url loader to work on both browser and utility processes
devm33 Sep 21, 2023
5976e4d
chore: add net files to utility process bundle
devm33 Sep 21, 2023
5d0ea84
chore: re-add app ready check but only on main process
devm33 Sep 21, 2023
f1628c3
chore: replace browser thread dcheck's with sequence checker
devm33 Sep 21, 2023
550dc83
refactor: move url loader from browser to common
devm33 Sep 21, 2023
55157d2
refactor: move net-client-request.ts from browser to common
devm33 Sep 22, 2023
383a817
docs: add utility process to net api docs
devm33 Sep 22, 2023
a017f9f
refactor: move net module app ready check to browser only
devm33 Sep 24, 2023
2c5de7a
refactor: switch import from main to common after moving to common
devm33 Sep 26, 2023
c9960ae
test: add basic net module test for utility process
devm33 Sep 27, 2023
aa0afa1
refactor: switch browser pid with utility pid
devm33 Sep 28, 2023
b52584f
refactor: move electron_api_net from browser to common
devm33 Sep 28, 2023
252133b
chore: add fetch to utility net module
devm33 Sep 29, 2023
457ba82
chore: add isOnline and online to utility net module
devm33 Sep 29, 2023
21f129d
refactor: move net spec helpers into helper file
devm33 Sep 30, 2023
8b16a37
refactor: break apart net module tests
devm33 Oct 1, 2023
a635a57
test: add utility process mocha runner to run net module tests
devm33 Oct 1, 2023
b82b236
docs: add utility process to net module classes
devm33 Oct 2, 2023
39856d6
refactor: update imports in lib/utility to use electron/utility
devm33 Oct 17, 2023
ae385f8
chore: check browser context before using in main process
devm33 Nov 21, 2023
805d33d
chore: remove test debugging
devm33 Nov 21, 2023
3bab188
chore: remove unnecessary header include
devm33 Dec 13, 2023
53bb415
docs: add utility process net module limitations
devm33 Dec 14, 2023
370e3c6
test: run net module tests in utility process individually
devm33 Dec 15, 2023
be73c6b
refactor: clean up prior utility process net tests
devm33 Dec 16, 2023
cc4fb8f
chore: add resolveHost to utility process net module
devm33 Dec 17, 2023
036a6f7
chore: replace resolve host dcheck with sequence checker
devm33 Dec 20, 2023
6fdf2ac
test: add net module tests for net.resolveHost
devm33 Dec 20, 2023
d62c765
docs: remove utility process limitation for resolveHost
devm33 Dec 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/api/client-request.md
Expand Up @@ -2,7 +2,7 @@

> Make HTTP/HTTPS requests.

Process: [Main](../glossary.md#main-process)<br />
Process: [Main](../glossary.md#main-process), [Utility](../glossary.md#utility-process)<br />
_This class is not exported from the `'electron'` module. It is only available as a return value of other methods in the Electron API._

`ClientRequest` implements the [Writable Stream](https://nodejs.org/api/stream.html#stream_writable_streams)
Expand Down
2 changes: 1 addition & 1 deletion docs/api/incoming-message.md
Expand Up @@ -2,7 +2,7 @@

> Handle responses to HTTP/HTTPS requests.

Process: [Main](../glossary.md#main-process)<br />
Process: [Main](../glossary.md#main-process), [Utility](../glossary.md#utility-process)<br />
_This class is not exported from the `'electron'` module. It is only available as a return value of other methods in the Electron API._

`IncomingMessage` implements the [Readable Stream](https://nodejs.org/api/stream.html#stream_readable_streams)
Expand Down
5 changes: 4 additions & 1 deletion docs/api/net.md
Expand Up @@ -2,7 +2,7 @@

> Issue HTTP/HTTPS requests using Chromium's native networking library

Process: [Main](../glossary.md#main-process)
Process: [Main](../glossary.md#main-process), [Utility](../glossary.md#utility-process)

The `net` module is a client-side API for issuing HTTP(S) requests. It is
similar to the [HTTP](https://nodejs.org/api/http.html) and
Expand Down Expand Up @@ -119,6 +119,9 @@ protocol.handle('https', (req) => {
})
```

Note: in the [utility process](../glossary.md#utility-process) custom protocols
are not supported.

### `net.isOnline()`

Returns `boolean` - Whether there is currently internet connection.
Expand Down
6 changes: 5 additions & 1 deletion filenames.auto.gni
Expand Up @@ -219,7 +219,6 @@ auto_filenames = {
"lib/browser/api/message-channel.ts",
"lib/browser/api/module-list.ts",
"lib/browser/api/native-theme.ts",
"lib/browser/api/net-client-request.ts",
"lib/browser/api/net-fetch.ts",
"lib/browser/api/net-log.ts",
"lib/browser/api/net.ts",
Expand Down Expand Up @@ -255,6 +254,7 @@ auto_filenames = {
"lib/browser/web-view-events.ts",
"lib/common/api/module-list.ts",
"lib/common/api/native-image.ts",
"lib/common/api/net-client-request.ts",
"lib/common/api/shell.ts",
"lib/common/define-properties.ts",
"lib/common/deprecate.ts",
Expand Down Expand Up @@ -348,12 +348,16 @@ auto_filenames = {
]

utility_bundle_deps = [
"lib/browser/api/net-fetch.ts",
"lib/browser/message-port-main.ts",
"lib/common/api/net-client-request.ts",
"lib/common/define-properties.ts",
"lib/common/init.ts",
"lib/common/reset-search-paths.ts",
"lib/common/webpack-globals-provider.ts",
"lib/utility/api/exports/electron.ts",
"lib/utility/api/module-list.ts",
"lib/utility/api/net.ts",
"lib/utility/init.ts",
"lib/utility/parent-port.ts",
"package.json",
Expand Down
6 changes: 3 additions & 3 deletions filenames.gni
Expand Up @@ -279,7 +279,6 @@ filenames = {
"shell/browser/api/electron_api_menu.h",
"shell/browser/api/electron_api_native_theme.cc",
"shell/browser/api/electron_api_native_theme.h",
"shell/browser/api/electron_api_net.cc",
"shell/browser/api/electron_api_net_log.cc",
"shell/browser/api/electron_api_net_log.h",
"shell/browser/api/electron_api_notification.cc",
Expand All @@ -305,8 +304,6 @@ filenames = {
"shell/browser/api/electron_api_system_preferences.h",
"shell/browser/api/electron_api_tray.cc",
"shell/browser/api/electron_api_tray.h",
"shell/browser/api/electron_api_url_loader.cc",
"shell/browser/api/electron_api_url_loader.h",
"shell/browser/api/electron_api_utility_process.cc",
"shell/browser/api/electron_api_utility_process.h",
"shell/browser/api/electron_api_view.cc",
Expand Down Expand Up @@ -544,8 +541,11 @@ filenames = {
"shell/common/api/electron_api_key_weak_map.h",
"shell/common/api/electron_api_native_image.cc",
"shell/common/api/electron_api_native_image.h",
"shell/common/api/electron_api_net.cc",
"shell/common/api/electron_api_shell.cc",
"shell/common/api/electron_api_testing.cc",
"shell/common/api/electron_api_url_loader.cc",
"shell/common/api/electron_api_url_loader.h",
"shell/common/api/electron_api_v8_util.cc",
"shell/common/api/electron_bindings.cc",
"shell/common/api/electron_bindings.h",
Expand Down
9 changes: 5 additions & 4 deletions lib/browser/api/net-fetch.ts
@@ -1,6 +1,6 @@
import { net, IncomingMessage, Session as SessionT } from 'electron/main';
import { ClientRequestConstructorOptions, ClientRequest, IncomingMessage, Session as SessionT } from 'electron/main';
import { Readable, Writable, isReadable } from 'stream';
import { allowAnyProtocol } from '@electron/internal/browser/api/net-client-request';
import { allowAnyProtocol } from '@electron/internal/common/api/net-client-request';

function createDeferredPromise<T, E extends Error = Error> (): { promise: Promise<T>; resolve: (x: T) => void; reject: (e: E) => void; } {
let res: (x: T) => void;
Expand All @@ -13,7 +13,8 @@ function createDeferredPromise<T, E extends Error = Error> (): { promise: Promis
return { promise, resolve: res!, reject: rej! };
}

export function fetchWithSession (input: RequestInfo, init: (RequestInit & {bypassCustomProtocolHandlers?: boolean}) | undefined, session: SessionT): Promise<Response> {
export function fetchWithSession (input: RequestInfo, init: (RequestInit & {bypassCustomProtocolHandlers?: boolean}) | undefined, session: SessionT | undefined,
request: (options: ClientRequestConstructorOptions | string) => ClientRequest) {
const p = createDeferredPromise<Response>();
let req: Request;
try {
Expand Down Expand Up @@ -73,7 +74,7 @@ export function fetchWithSession (input: RequestInfo, init: (RequestInit & {bypa
// We can't set credentials to same-origin unless there's an origin set.
const credentials = req.credentials === 'same-origin' && !origin ? 'include' : req.credentials;

const r = net.request(allowAnyProtocol({
const r = request(allowAnyProtocol({
session,
method: req.method,
url: req.url,
Expand Down
9 changes: 6 additions & 3 deletions lib/browser/api/net.ts
@@ -1,10 +1,13 @@
import { IncomingMessage, session } from 'electron/main';
import { app, IncomingMessage, session } from 'electron/main';
import type { ClientRequestConstructorOptions } from 'electron/main';
import { ClientRequest } from '@electron/internal/browser/api/net-client-request';
import { ClientRequest } from '@electron/internal/common/api/net-client-request';

const { isOnline } = process._linkedBinding('electron_browser_net');
const { isOnline } = process._linkedBinding('electron_common_net');

export function request (options: ClientRequestConstructorOptions | string, callback?: (message: IncomingMessage) => void) {
if (!app.isReady()) {
throw new Error('net module can only be used after app is ready');
}
return new ClientRequest(options, callback);
}

Expand Down
3 changes: 2 additions & 1 deletion lib/browser/api/session.ts
@@ -1,8 +1,9 @@
import { fetchWithSession } from '@electron/internal/browser/api/net-fetch';
import { net } from 'electron/main';
const { fromPartition, fromPath, Session } = process._linkedBinding('electron_browser_session');

Session.prototype.fetch = function (input: RequestInfo, init?: RequestInit) {
return fetchWithSession(input, init, this);
return fetchWithSession(input, init, this, net.request);
};

export default {
Expand Down
2 changes: 1 addition & 1 deletion lib/browser/guest-view-manager.ts
Expand Up @@ -14,7 +14,7 @@ interface GuestInstance {
}

const webViewManager = process._linkedBinding('electron_browser_web_view_manager');
const netBinding = process._linkedBinding('electron_browser_net');
const netBinding = process._linkedBinding('electron_common_net');

const supportedWebViewEvents = Object.keys(webViewEvents);

Expand Down
@@ -1,14 +1,15 @@
import * as url from 'url';
import { Readable, Writable } from 'stream';
import { app } from 'electron/main';
import type { ClientRequestConstructorOptions, UploadProgress } from 'electron/main';
import type {
ClientRequestConstructorOptions,
UploadProgress
} from 'electron/common';

const {
isValidHeaderName,
isValidHeaderValue,
createURLLoader
} = process._linkedBinding('electron_browser_net');
const { Session } = process._linkedBinding('electron_browser_session');
} = process._linkedBinding('electron_common_net');

const kHttpProtocols = new Set(['http:', 'https:']);

Expand Down Expand Up @@ -283,14 +284,17 @@ function parseOptions (optionsIn: ClientRequestConstructorOptions | string): Nod
const key = name.toLowerCase();
urlLoaderOptions.headers[key] = { name, value };
}
if (options.session) {
if (!(options.session instanceof Session)) { throw new TypeError('`session` should be an instance of the Session class'); }
urlLoaderOptions.session = options.session;
} else if (options.partition) {
if (typeof options.partition === 'string') {
urlLoaderOptions.partition = options.partition;
} else {
throw new TypeError('`partition` should be a string');
if (process.type !== 'utility') {
const { Session } = process._linkedBinding('electron_browser_session');
if (options.session) {
if (!(options.session instanceof Session)) { throw new TypeError('`session` should be an instance of the Session class'); }
urlLoaderOptions.session = options.session;
} else if (options.partition) {
if (typeof options.partition === 'string') {
urlLoaderOptions.partition = options.partition;
} else {
throw new TypeError('`partition` should be a string');
}
}
}
return urlLoaderOptions;
Expand All @@ -312,10 +316,6 @@ export class ClientRequest extends Writable implements Electron.ClientRequest {
constructor (options: ClientRequestConstructorOptions | string, callback?: (message: IncomingMessage) => void) {
super({ autoDestroy: true });

if (!app.isReady()) {
throw new Error('net module can only be used after app is ready');
}

if (callback) {
this.once('response', callback);
}
Expand Down
4 changes: 3 additions & 1 deletion lib/utility/api/module-list.ts
@@ -1,2 +1,4 @@
// Utility side modules, please sort alphabetically.
export const utilityNodeModuleList: ElectronInternal.ModuleEntry[] = [];
export const utilityNodeModuleList: ElectronInternal.ModuleEntry[] = [
{ name: 'net', loader: () => require('./net') }
];
22 changes: 22 additions & 0 deletions lib/utility/api/net.ts
@@ -0,0 +1,22 @@
import { IncomingMessage } from 'electron/utility';
import type { ClientRequestConstructorOptions } from 'electron/utility';
import { ClientRequest } from '@electron/internal/common/api/net-client-request';
import { fetchWithSession } from '@electron/internal/browser/api/net-fetch';

const { isOnline, resolveHost } = process._linkedBinding('electron_common_net');

export function request (options: ClientRequestConstructorOptions | string, callback?: (message: IncomingMessage) => void) {
return new ClientRequest(options, callback);
}

export function fetch (input: RequestInfo, init?: RequestInit): Promise<Response> {
return fetchWithSession(input, init, undefined, request);
}

exports.resolveHost = resolveHost;

exports.isOnline = isOnline;

Object.defineProperty(exports, 'online', {
get: () => isOnline()
});
3 changes: 3 additions & 0 deletions lib/utility/init.ts
@@ -1,3 +1,4 @@
import { EventEmitter } from 'events';
import { pathToFileURL } from 'url';

import { ParentPort } from '@electron/internal/utility/parent-port';
Expand All @@ -15,6 +16,8 @@ require('../common/reset-search-paths');
// Import common settings.
require('@electron/internal/common/init');

process._linkedBinding('electron_browser_event_emitter').setEventEmitterPrototype(EventEmitter.prototype);

const parentPort: ParentPort = new ParentPort();
Object.defineProperty(process, 'parentPort', {
enumerable: true,
Expand Down
18 changes: 18 additions & 0 deletions shell/browser/api/electron_api_utility_process.cc
Expand Up @@ -13,6 +13,7 @@
#include "base/process/kill.h"
#include "base/process/launch.h"
#include "base/process/process.h"
#include "chrome/browser/browser_process.h"
nornagon marked this conversation as resolved.
Show resolved Hide resolved
#include "content/public/browser/child_process_host.h"
#include "content/public/browser/service_process_host.h"
#include "content/public/common/result_codes.h"
Expand All @@ -22,6 +23,7 @@
#include "mojo/public/cpp/bindings/pending_receiver.h"
#include "shell/browser/api/message_port.h"
#include "shell/browser/javascript_environment.h"
#include "shell/browser/net/system_network_context_manager.h"
#include "shell/common/gin_converters/callback_converter.h"
#include "shell/common/gin_converters/file_path_converter.h"
#include "shell/common/gin_helper/dictionary.h"
Expand Down Expand Up @@ -192,6 +194,22 @@ UtilityProcessWrapper::UtilityProcessWrapper(
connector_->set_connection_error_handler(base::BindOnce(
&UtilityProcessWrapper::CloseConnectorPort, weak_factory_.GetWeakPtr()));

mojo::PendingRemote<network::mojom::URLLoaderFactory> url_loader_factory;
network::mojom::URLLoaderFactoryParamsPtr loader_params =
network::mojom::URLLoaderFactoryParams::New();
loader_params->process_id = pid_;
loader_params->is_corb_enabled = false;
loader_params->is_trusted = true;
network::mojom::NetworkContext* network_context =
g_browser_process->system_network_context_manager()->GetContext();
network_context->CreateURLLoaderFactory(
url_loader_factory.InitWithNewPipeAndPassReceiver(),
std::move(loader_params));
params->url_loader_factory = std::move(url_loader_factory);
mojo::PendingRemote<network::mojom::HostResolver> host_resolver;
network_context->CreateHostResolver(
{}, host_resolver.InitWithNewPipeAndPassReceiver());
params->host_resolver = std::move(host_resolver);
node_service_remote_->Initialize(std::move(params));
}

Expand Down
34 changes: 24 additions & 10 deletions shell/browser/net/resolve_host_function.cc
Expand Up @@ -15,7 +15,10 @@
#include "net/base/net_errors.h"
#include "net/base/network_isolation_key.h"
#include "net/dns/public/resolve_error_info.h"
#include "services/network/public/mojom/network_context.mojom.h"
#include "shell/browser/electron_browser_context.h"
#include "shell/common/process_util.h"
#include "shell/services/node/node_service.h"
#include "url/origin.h"

using content::BrowserThread;
Expand All @@ -30,15 +33,17 @@ ResolveHostFunction::ResolveHostFunction(
: browser_context_(browser_context),
host_(std::move(host)),
params_(std::move(params)),
callback_(std::move(callback)) {}
callback_(std::move(callback)) {
DETACH_FROM_SEQUENCE(sequence_checker_);
}

ResolveHostFunction::~ResolveHostFunction() {
DCHECK_CURRENTLY_ON(BrowserThread::UI);
DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
DCHECK(!receiver_.is_bound());
}

void ResolveHostFunction::Run() {
DCHECK_CURRENTLY_ON(BrowserThread::UI);
DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);
DCHECK(!receiver_.is_bound());

// Start the request.
Expand All @@ -50,12 +55,21 @@ void ResolveHostFunction::Run() {
net::ResolveErrorInfo(net::ERR_FAILED),
/*resolved_addresses=*/absl::nullopt,
/*endpoint_results_with_metadata=*/absl::nullopt));
browser_context_->GetDefaultStoragePartition()
->GetNetworkContext()
->ResolveHost(network::mojom::HostResolverHost::NewHostPortPair(
std::move(host_port_pair)),
net::NetworkAnonymizationKey(), std::move(params_),
std::move(resolve_host_client));
if (electron::IsUtilityProcess()) {
URLLoaderBundle::GetInstance()->GetHostResolver()->ResolveHost(
network::mojom::HostResolverHost::NewHostPortPair(
std::move(host_port_pair)),
net::NetworkAnonymizationKey(), std::move(params_),
std::move(resolve_host_client));
} else {
DCHECK_CURRENTLY_ON(BrowserThread::UI);
browser_context_->GetDefaultStoragePartition()
->GetNetworkContext()
->ResolveHost(network::mojom::HostResolverHost::NewHostPortPair(
std::move(host_port_pair)),
net::NetworkAnonymizationKey(), std::move(params_),
std::move(resolve_host_client));
}
}

void ResolveHostFunction::OnComplete(
Expand All @@ -64,7 +78,7 @@ void ResolveHostFunction::OnComplete(
const absl::optional<net::AddressList>& resolved_addresses,
const absl::optional<net::HostResolverEndpointResults>&
endpoint_results_with_metadata) {
DCHECK_CURRENTLY_ON(BrowserThread::UI);
DCHECK_CALLED_ON_VALID_SEQUENCE(sequence_checker_);

// Ensure that we outlive the `receiver_.reset()` call.
scoped_refptr<ResolveHostFunction> self(this);
Expand Down
3 changes: 3 additions & 0 deletions shell/browser/net/resolve_host_function.h
Expand Up @@ -9,6 +9,7 @@

#include "base/memory/raw_ptr.h"
#include "base/memory/ref_counted.h"
#include "base/sequence_checker.h"
#include "mojo/public/cpp/bindings/receiver.h"
#include "net/base/address_list.h"
#include "net/dns/public/host_resolver_results.h"
Expand Down Expand Up @@ -53,6 +54,8 @@ class ResolveHostFunction
const absl::optional<net::HostResolverEndpointResults>&
endpoint_results_with_metadata) override;

SEQUENCE_CHECKER(sequence_checker_);

// Receiver for the currently in-progress request, if any.
mojo::Receiver<network::mojom::ResolveHostClient> receiver_{this};

Expand Down