[BestEffortServiceWorker] Dedupe main resource

This CL is a part of efforts that achieve the dedupe function to
BestEffortServiceWorker.

At a high-level, the dedupe is achieved by reusing the
RaceNetworkRequest response as a response of the corresponding request
in the fetch handler, and do not send a new fetch request.

We can split into some smaller steps to implement this.
1. Generates a token of RaceNetworkRequest, and pass it to the
blink side to make the corresponding request in the fetch handler
discoverable.
2. Adds a map of the token and URLLoaderFactory in
ServiceWorkerGlobalScope, and replaces the default URLLoader when the
fetch request is same as the one called in RaceNetworkRequest which is
outside of the fetch handler.
3. (This CL) Fuses two message pipes into one in a custom
URLLoaderClient so that we can reuse the RaceNetworkRequest result and
don't dispatch duplicated requests in the fetch handler.

This CL introduces one new class.
`ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory` is the
URLLoaderFactory that fuses the RaceNetworkRequest URLLoaderClient mojo
message pipe and the other URLLoaderClient pipe for the corresponding
fetch request in the ServiceWorker fetch handler in blink side. By
fusing them into one message pipe, the request in the fetch handler
never creates a new URLLoader and thus no duplicated request, and reuses
the response of RaceNetworkRequest.

`ServiceWorkerRaceNetworkRequestDataPipeManager` is the class managing
the dedicated data pipe and buffering data from RaceNetworkRequset. We
use this class to create new data pipes and pass the response of
original RaceNetworkRequset to the RaceNetworkRequest itself and the
fetch handler, respectively.

`ServiceWorkerRaceNetworkRequestURLLoaderClient` is changed to send the
response data to newly created two mojo data pipes, one is for the
RaceNetworkRequest itself, and the other one is for the fetch handler.

This CL adds the dedupe feature to the main resource only. For
subresoureces, we don't support it at least for now due to the
complexity of redirect case, but the ActiveEntry in net layer would
dedupe requests in most cases in reality.

Here is a design doc including diagram how mojo pipes work.
https://docs.google.com/document/d/1eotFetBW9RUE2sYjCnVl7P8rbvIsWBJcy3NOswC5kek/edit#bookmark=id.umdo7cpwavad

Change-Id: Ia524f69a81335528a66db5df2488a1f8190f2024
Bug: 1420517
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4560732
Reviewed-by: Rakina Zata Amni <rakina@chromium.org>
Commit-Queue: Shunya Shishido <sisidovski@chromium.org>
Reviewed-by: Minoru Chikamune <chikamune@chromium.org>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Reviewed-by: Tsuyoshi Horo <horo@chromium.org>
Reviewed-by: Kouhei Ueno <kouhei@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1158698}

content/browser/service_worker/service_worker_browsertest.cc

@@ -5297,7 +5297,7 @@ class ServiceWorkerRaceNetworkRequestBrowserTest
     return web_contents()->GetPrimaryMainFrame();
   }
 
-  void SetupAndRegisterServiceWorker() {
+  scoped_refptr<ServiceWorkerVersion> SetupAndRegisterServiceWorker() {
     RegisterRequestMonitorForRequestCount();
     RegisterRequestHandlerForSlowResponsePage();
     StartServerAndNavigateToSetup();
@@ -5307,18 +5307,20 @@ class ServiceWorkerRaceNetworkRequestBrowserTest
 
     // Register a service worker.
     WorkerRunningStatusObserver observer1(public_context());
-    ASSERT_TRUE(NavigateToURL(shell(), create_service_worker_url));
-    ASSERT_EQ("DONE",
+    EXPECT_TRUE(NavigateToURL(shell(), create_service_worker_url));
+    EXPECT_EQ("DONE",
               EvalJs(GetPrimaryMainFrame(),
                      "register('/service_worker/race_network_request.js')"));
     observer1.WaitUntilRunning();
     scoped_refptr<ServiceWorkerVersion> version =
         wrapper()->GetLiveVersion(observer1.version_id());
-    ASSERT_EQ(EmbeddedWorkerStatus::RUNNING, version->running_status());
+    EXPECT_EQ(EmbeddedWorkerStatus::RUNNING, version->running_status());
 
     // Stop the current running service worker.
     StopServiceWorker(version.get());
-    ASSERT_EQ(EmbeddedWorkerStatus::STOPPED, version->running_status());
+    EXPECT_EQ(EmbeddedWorkerStatus::STOPPED, version->running_status());
+
+    return version;
   }
 
   EvalJsResult GetInnerText() {
@@ -5599,6 +5601,64 @@ IN_PROC_BROWSER_TEST_F(ServiceWorkerRaceNetworkRequestBrowserTest,
             GetInnerText());
 }
 
+IN_PROC_BROWSER_TEST_F(ServiceWorkerRaceNetworkRequestBrowserTest,
+                       FetchHandler_PassThrough) {
+  // Register the ServiceWorker and navigate to the in scope URL.
+  scoped_refptr<ServiceWorkerVersion> version = SetupAndRegisterServiceWorker();
+  // Capture the response head.
+  const std::string relative_url =
+      "/service_worker/mock_response?sw_pass_through";
+  const GURL test_url = embedded_test_server()->GetURL(relative_url);
+
+  WorkerRunningStatusObserver service_worker_running_status_observer(
+      public_context());
+  NavigationHandleObserver observer(web_contents(), test_url);
+  NavigateToURLBlockUntilNavigationsComplete(shell(), test_url, 1);
+  EXPECT_TRUE(observer.has_committed());
+  service_worker_running_status_observer.WaitUntilRunning();
+
+  // Request count should be 1. RaceNetworkRequest + pass through request from
+  // fetch handler but the fetch handler request will reuse the response from
+  // RaceNetworkRequest.
+  //
+  // TODO(crbug.com/1420517) Add the mechanism to wait for the fetch handler
+  // completion signal to ensure the request count is exactly not incremented
+  // anymore. Currently we don't record the UMA for the fetch handler completion
+  // if the RaceNetworkRequest wins.
+  while (GetRequestCount(relative_url) != 1) {
+    base::RunLoop().RunUntilIdle();
+  }
+  EXPECT_EQ(1, GetRequestCount(relative_url));
+}
+
+IN_PROC_BROWSER_TEST_F(ServiceWorkerRaceNetworkRequestBrowserTest,
+                       FetchHandler_PassThrough_Clone) {
+  // Register the ServiceWorker and navigate to the in scope URL.
+  scoped_refptr<ServiceWorkerVersion> version = SetupAndRegisterServiceWorker();
+  // URL which create a cloned request and pass-through.
+  const std::string relative_url_for_clone =
+      "/service_worker/mock_response?sw_clone_pass_through";
+  const GURL test_url_for_clone =
+      embedded_test_server()->GetURL(relative_url_for_clone);
+
+  WorkerRunningStatusObserver service_worker_running_status_observer(
+      public_context());
+  NavigationHandleObserver observer_for_clone(web_contents(),
+                                              test_url_for_clone);
+  NavigateToURLBlockUntilNavigationsComplete(shell(), test_url_for_clone, 1);
+  EXPECT_TRUE(observer_for_clone.has_committed());
+  service_worker_running_status_observer.WaitUntilRunning();
+
+  // Request count should be 2. RaceNetworkRequest + pass through cloned request
+  // from fetch handler. The fetch handler will create a new request because the
+  // request is cloned hence it may have different metadata from the one
+  // initiated by RaceNetworkRequest.
+  while (GetRequestCount(relative_url_for_clone) != 2) {
+    base::RunLoop().RunUntilIdle();
+  }
+  EXPECT_EQ(2, GetRequestCount(relative_url_for_clone));
+}
+
 // TODO(crbug.com/1431421): Flaky on Fuchsia.
 #if BUILDFLAG(IS_FUCHSIA)
 #define MAYBE_Subresource_NetworkRequest_Wins \

content/browser/service_worker/service_worker_fetch_dispatcher.cc

@@ -669,6 +669,10 @@ void ServiceWorkerFetchDispatcher::DispatchFetchEvent() {
     params->request->service_worker_race_network_request_token =
         race_network_request_token_;
   }
+  if (race_network_request_loader_factory_) {
+    params->race_network_request_loader_factory =
+        std::move(race_network_request_loader_factory_);
+  }
 
   // |endpoint()| is owned by |version_|. So it is safe to pass the
   // unretained raw pointer of |version_| to OnFetchEventFinished callback.

content/browser/service_worker/service_worker_fetch_dispatcher.h

@@ -92,6 +92,10 @@ class CONTENT_EXPORT ServiceWorkerFetchDispatcher {
   void set_race_network_request_token(base::UnguessableToken token) {
     race_network_request_token_ = token;
   }
+  void set_race_network_request_loader_factory(
+      mojo::PendingRemote<network::mojom::URLLoaderFactory> factory) {
+    race_network_request_loader_factory_ = std::move(factory);
+  }
 
  private:
   class ResponseCallback;
@@ -146,6 +150,8 @@ class CONTENT_EXPORT ServiceWorkerFetchDispatcher {
       preload_url_loader_client_receiver_;
 
   base::UnguessableToken race_network_request_token_;
+  mojo::PendingRemote<network::mojom::URLLoaderFactory>
+      race_network_request_loader_factory_;
 
   // Whether to dispatch an offline-capability-check fetch event.
   const bool is_offline_capability_check_ = false;

content/browser/service_worker/service_worker_main_resource_loader.cc

@@ -275,13 +275,40 @@ bool ServiceWorkerMainResourceLoader::MaybeStartRaceNetworkRequest(
     return false;
   }
 
+  // RaceNetworkRequest is triggered only in a main frame.
+  if (resource_request_.destination !=
+      network::mojom::RequestDestination::kDocument) {
+    return false;
+  }
+
   // Create URLLoader related assets to handle the request triggered by
   // RaceNetworkRequset.
+  mojo::PendingRemote<network::mojom::URLLoaderClient> forwarding_client;
+  forwarded_race_network_request_url_loader_factory_ = std::make_unique<
+      ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory>(
+      forwarding_client.InitWithNewPipeAndPassReceiver(),
+      resource_request_.url);
+  auto race_network_request_url_loader_client = std::make_unique<
+      ServiceWorkerRaceNetworkRequestURLLoaderClient>(
+      resource_request_, AsWeakPtr(), std::move(forwarding_client),
+      network::features::GetDataPipeDefaultAllocationSize(
+          network::features::DataPipeAllocationSize::kLargerSizeIfPossible));
+
+  // If the initial state is not kWaitForBody, that means creating data pipes
+  // failed. Do not start RaceNetworkRequest this case.
+  if (race_network_request_url_loader_client->state() !=
+      ServiceWorkerRaceNetworkRequestURLLoaderClient::State::kWaitForBody) {
+    return false;
+  }
+
+  mojo::PendingRemote<network::mojom::URLLoaderFactory> remote_factory;
+  forwarded_race_network_request_url_loader_factory_->Clone(
+      remote_factory.InitWithNewPipeAndPassReceiver());
   fetch_dispatcher_->set_race_network_request_token(
       base::UnguessableToken::Create());
-  auto race_network_request_url_loader_client =
-      std::make_unique<ServiceWorkerRaceNetworkRequestURLLoaderClient>(
-          resource_request_, AsWeakPtr());
+  fetch_dispatcher_->set_race_network_request_loader_factory(
+      std::move(remote_factory));
+
   mojo::PendingRemote<network::mojom::URLLoaderClient> client_to_pass;
   race_network_request_url_loader_client->Bind(&client_to_pass);
   scoped_refptr<network::SharedURLLoaderFactory> factory =

content/browser/service_worker/service_worker_main_resource_loader.h

@@ -16,6 +16,7 @@
 #include "content/browser/service_worker/embedded_worker_status.h"
 #include "content/browser/service_worker/service_worker_fetch_dispatcher.h"
 #include "content/common/content_export.h"
+#include "content/common/service_worker/forwarded_race_network_request_url_loader_factory.h"
 #include "content/common/service_worker/race_network_request_url_loader_client.h"
 #include "content/common/service_worker/service_worker_resource_loader.h"
 #include "mojo/public/cpp/bindings/pending_receiver.h"
@@ -282,6 +283,8 @@ class CONTENT_EXPORT ServiceWorkerMainResourceLoader
       race_network_request_url_loader_;
   std::unique_ptr<ServiceWorkerRaceNetworkRequestURLLoaderClient>
       race_network_request_loader_client_;
+  std::unique_ptr<ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory>
+      forwarded_race_network_request_url_loader_factory_;
 
   base::WeakPtrFactory<ServiceWorkerMainResourceLoader> weak_factory_{this};
 };

content/common/BUILD.gn

@@ -146,6 +146,8 @@ source_set("common") {
     "process_visibility_tracker.h",
     "pseudonymization_salt.cc",
     "pseudonymization_salt.h",
+    "service_worker/forwarded_race_network_request_url_loader_factory.cc",
+    "service_worker/forwarded_race_network_request_url_loader_factory.h",
     "service_worker/race_network_request_url_loader_client.cc",
     "service_worker/race_network_request_url_loader_client.h",
     "service_worker/service_worker_resource_loader.cc",

content/common/service_worker/forwarded_race_network_request_url_loader_factory.cc

@@ -0,0 +1,35 @@
+// Copyright 2023 The Chromium Authors
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#include "content/common/service_worker/forwarded_race_network_request_url_loader_factory.h"
+
+namespace content {
+
+ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory::
+    ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory(
+        mojo::PendingReceiver<network::mojom::URLLoaderClient> client_receiver,
+        const GURL& url)
+    : client_receiver_(std::move(client_receiver)), url_(url) {}
+
+ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory::
+    ~ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory() = default;
+
+void ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory::
+    CreateLoaderAndStart(
+        mojo::PendingReceiver<network::mojom::URLLoader> receiver,
+        int32_t request_id,
+        uint32_t options,
+        const network::ResourceRequest& resource_request,
+        mojo::PendingRemote<network::mojom::URLLoaderClient> client,
+        const net::MutableNetworkTrafficAnnotationTag& traffic_annotation) {
+  CHECK_EQ(url_, resource_request.url);
+  bool result = mojo::FusePipes(std::move(client_receiver_), std::move(client));
+  CHECK(result);
+}
+
+void ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory::Clone(
+    mojo::PendingReceiver<network::mojom::URLLoaderFactory> receiver) {
+  receiver_.Bind(std::move(receiver));
+}
+}  // namespace content

content/common/service_worker/forwarded_race_network_request_url_loader_factory.h

@@ -0,0 +1,55 @@
+// Copyright 2023 The Chromium Authors
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CONTENT_COMMON_SERVICE_WORKER_FORWARDED_RACE_NETWORK_REQUEST_URL_LOADER_FACTORY_H_
+#define CONTENT_COMMON_SERVICE_WORKER_FORWARDED_RACE_NETWORK_REQUEST_URL_LOADER_FACTORY_H_
+
+#include "mojo/public/cpp/bindings/pending_receiver.h"
+#include "mojo/public/cpp/bindings/receiver.h"
+#include "mojo/public/cpp/bindings/remote.h"
+#include "services/network/public/cpp/resource_request.h"
+#include "services/network/public/cpp/shared_url_loader_factory.h"
+#include "services/network/public/mojom/url_loader.mojom.h"
+#include "services/network/public/mojom/url_loader_factory.mojom.h"
+
+namespace content {
+// A URLLoaderFactory for BestEffortServiceWorker (crbug.com/1420517).
+// RaceNetworkRequest is initiated outside of ServiceWorker, but the response
+// will be reused as a corresponding fetch event result in ServiceWorker in
+// order to avoid sending duplicated requests.
+// This URLLoaderFactory fuses two different message pipes into a single pipe by
+// passing |client_receiver| in the constructor and calling
+// CreateLoaderAndStart().
+class ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory
+    : network::mojom::URLLoaderFactory {
+ public:
+  ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory(
+      mojo::PendingReceiver<network::mojom::URLLoaderClient> client_receiver,
+      const GURL& url);
+  ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory(
+      const ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory&) = delete;
+  ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory& operator=(
+      const ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory&) = delete;
+  ~ServiceWorkerForwardedRaceNetworkRequestURLLoaderFactory() override;
+
+  // network::mojom::URLLoaderFactory:
+  void CreateLoaderAndStart(
+      mojo::PendingReceiver<network::mojom::URLLoader> receiver,
+      int32_t request_id,
+      uint32_t options,
+      const network::ResourceRequest& resource_request,
+      mojo::PendingRemote<network::mojom::URLLoaderClient> client,
+      const net::MutableNetworkTrafficAnnotationTag& traffic_annotation)
+      override;
+  void Clone(mojo::PendingReceiver<network::mojom::URLLoaderFactory> receiver)
+      override;
+
+ private:
+  mojo::Receiver<network::mojom::URLLoaderFactory> receiver_{this};
+  mojo::PendingReceiver<network::mojom::URLLoaderClient> client_receiver_;
+  GURL url_;
+};
+}  // namespace content
+
+#endif  // CONTENT_COMMON_SERVICE_WORKER_FORWARDED_RACE_NETWORK_REQUEST_URL_LOADER_FACTORY_H_