Use no-cache fetch mode when loading main documents with location.rel…

…oad()

https://bugs.webkit.org/show_bug.cgi?id=181285
LayoutTests/imported/w3c:

Patch by Youenn Fablet <youenn@apple.com> on 2018-01-10
Reviewed by Alex Christensen.

* web-platform-tests/service-workers/service-worker/fetch-event.https-expected.txt:

Source/WebCore:

Patch by Youenn Fablet <youenn@apple.com> on 2018-01-10
Reviewed by Alex Christensen.

Covered by rebased tests.

Start to translate cache policy used for navigation as FetchOptions::Cache.
This allows ensuring service workers receive the right cache mode when intercepting navigation loads.
To not change current navigation behavior, ReturnCacheDataElseLoad and ReturnCacheDataDontLoad still trigger default fetch cache mode.

For Reload and ReloadExpiredOnly frame load types, using no-cache mode is more efficient than reload mode,
as a conditional request will be sent if possible. This applies to location.reload which is consistent with other browsers.
Keep reload mode for ReloadFromOrigin.

* loader/DocumentLoader.cpp:
(WebCore::toFetchOptionsCache):
(WebCore::DocumentLoader::loadMainResource):
* loader/FrameLoader.cpp:
(WebCore::FrameLoader::loadFrameRequest):
(WebCore::FrameLoader::loadURL):
(WebCore::FrameLoader::load):
(WebCore::FrameLoader::reload):
(WebCore::FrameLoader::defaultRequestCachingPolicy):
(WebCore::FrameLoader::loadDifferentDocumentItem):
* loader/NavigationScheduler.cpp:

LayoutTests:

<rdar://problem/36356831>

Patch by Youenn Fablet <youenn@apple.com> on 2018-01-10
Reviewed by Alex Christensen.

* http/tests/inspector/network/har/har-page-expected.txt:
* http/tests/inspector/network/har/har-page.html:

Canonical link: https://commits.webkit.org/197362@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@226745 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog

@@ -1,3 +1,14 @@
+2018-01-10  Youenn Fablet  <youenn@apple.com>
+
+        Use no-cache fetch mode when loading main documents with location.reload()
+        https://bugs.webkit.org/show_bug.cgi?id=181285
+        <rdar://problem/36356831>
+
+        Reviewed by Alex Christensen.
+
+        * http/tests/inspector/network/har/har-page-expected.txt:
+        * http/tests/inspector/network/har/har-page.html:
+
 2018-01-10  Per Arne Vollan  <pvollan@apple.com>
 
         Mark accessibility/table-header-calculation-for-header-rows.html as failure on Windows.

LayoutTests/http/tests/inspector/network/har/har-page-expected.txt

@@ -43,10 +43,10 @@ HAR Page Test.
           "cookies": [],
           "headers": "<filtered>",
           "content": {
-            "size": 3042,
+            "size": "<filtered>",
             "compression": 0,
             "mimeType": "text/html",
-            "text": "<filtered text (3042)>"
+            "text": "<filtered text (3171)>"
           },
           "redirectURL": "",
           "headersSize": "<filtered>",
@@ -84,7 +84,7 @@ HAR Page Test.
           "cookies": [],
           "headers": "<filtered>",
           "content": {
-            "size": 0,
+            "size": "<filtered>",
             "compression": 0,
             "mimeType": "application/x-javascript",
             "text": "<filtered text (7039)>"

LayoutTests/http/tests/inspector/network/har/har-page.html

@@ -42,6 +42,9 @@
         // Reduce the file contents.
         if (key === "text")
             return "<filtered text (" + value.length + ")>";
+        // Reduce the file size since cache may or may not be used.
+        if (key === "size")
+            return "<filtered>";
 
         // Since cache may or may not be used, timing data may be variable.
         // NOTE: SSL should always be -1 for this test case.

LayoutTests/imported/w3c/ChangeLog

@@ -1,3 +1,12 @@
+2018-01-10  Youenn Fablet  <youenn@apple.com>
+
+        Use no-cache fetch mode when loading main documents with location.reload()
+        https://bugs.webkit.org/show_bug.cgi?id=181285
+
+        Reviewed by Alex Christensen.
+
+        * web-platform-tests/service-workers/service-worker/fetch-event.https-expected.txt:
+
 2018-01-09  Chris Dumez  <cdumez@apple.com>
 
         We should not return undefined for most properties of a detached Window

LayoutTests/imported/w3c/web-platform-tests/service-workers/service-worker/fetch-event.https-expected.txt

@@ -1,5 +1,4 @@
 
-
 PASS Service Worker headers in the request of a fetch event 
 PASS Service Worker responds to fetch event with string 
 PASS Service Worker responds to fetch event with blob body 
@@ -12,7 +11,7 @@ PASS Service Worker responds to fetch event with POST form
 PASS Multiple calls of respondWith must throw InvalidStateErrors 
 PASS Service Worker event.respondWith must set the used flag 
 PASS Service Worker should expose FetchEvent URL fragments. 
-FAIL Service Worker responds to fetch event with the correct cache types assert_unreached: unexpected rejection: assert_equals: expected "no-cache" but got "default" Reached unreachable code
+PASS Service Worker responds to fetch event with the correct cache types 
 PASS Service Worker should intercept EventSource 
 PASS Service Worker responds to fetch event with the correct integrity_metadata 
 PASS FetchEvent#body is a string 

Source/WebCore/ChangeLog

@@ -1,3 +1,32 @@
+2018-01-10  Youenn Fablet  <youenn@apple.com>
+
+        Use no-cache fetch mode when loading main documents with location.reload()
+        https://bugs.webkit.org/show_bug.cgi?id=181285
+
+        Reviewed by Alex Christensen.
+
+        Covered by rebased tests.
+
+        Start to translate cache policy used for navigation as FetchOptions::Cache.
+        This allows ensuring service workers receive the right cache mode when intercepting navigation loads.
+        To not change current navigation behavior, ReturnCacheDataElseLoad and ReturnCacheDataDontLoad still trigger default fetch cache mode.
+
+        For Reload and ReloadExpiredOnly frame load types, using no-cache mode is more efficient than reload mode,
+        as a conditional request will be sent if possible. This applies to location.reload which is consistent with other browsers.
+        Keep reload mode for ReloadFromOrigin.
+
+        * loader/DocumentLoader.cpp:
+        (WebCore::toFetchOptionsCache):
+        (WebCore::DocumentLoader::loadMainResource):
+        * loader/FrameLoader.cpp:
+        (WebCore::FrameLoader::loadFrameRequest):
+        (WebCore::FrameLoader::loadURL):
+        (WebCore::FrameLoader::load):
+        (WebCore::FrameLoader::reload):
+        (WebCore::FrameLoader::defaultRequestCachingPolicy):
+        (WebCore::FrameLoader::loadDifferentDocumentItem):
+        * loader/NavigationScheduler.cpp:
+
 2018-01-10  Commit Queue  <commit-queue@webkit.org>
 
         Unreviewed, rolling out r226667 and r226673.

Source/WebCore/loader/DocumentLoader.cpp

@@ -1607,9 +1607,32 @@ void DocumentLoader::startLoadingMainResource()
     });
 }
 
+static inline FetchOptions::Cache toFetchOptionsCache(ResourceRequestCachePolicy policy)
+{
+    // We are setting FetchOptions::Cache values to keep current behavior consistency.
+    // FIXME: We should merge FetchOptions::Cache with ResourceRequestCachePolicy and merge related class members.
+    switch (policy) {
+    case UseProtocolCachePolicy:
+        return FetchOptions::Cache::Default;
+    case ReloadIgnoringCacheData:
+        return FetchOptions::Cache::Reload;
+    case ReturnCacheDataElseLoad:
+        return FetchOptions::Cache::Default;
+    case ReturnCacheDataDontLoad:
+        return FetchOptions::Cache::Default;
+    case DoNotUseAnyCache:
+        return FetchOptions::Cache::NoStore;
+    case RefreshAnyCacheData:
+        return FetchOptions::Cache::NoCache;
+    }
+    return FetchOptions::Cache::Default;
+}
+
 void DocumentLoader::loadMainResource(ResourceRequest&& request)
 {
-    static NeverDestroyed<ResourceLoaderOptions> mainResourceLoadOptions(SendCallbacks, SniffContent, BufferData, StoredCredentialsPolicy::Use, ClientCredentialPolicy::MayAskClientForCredentials, FetchOptions::Credentials::Include, SkipSecurityCheck, FetchOptions::Mode::Navigate, IncludeCertificateInfo, ContentSecurityPolicyImposition::SkipPolicyCheck, DefersLoadingPolicy::AllowDefersLoading, CachingPolicy::AllowCaching);
+    ResourceLoaderOptions mainResourceLoadOptions { SendCallbacks, SniffContent, BufferData, StoredCredentialsPolicy::Use, ClientCredentialPolicy::MayAskClientForCredentials, FetchOptions::Credentials::Include, SkipSecurityCheck, FetchOptions::Mode::Navigate, IncludeCertificateInfo, ContentSecurityPolicyImposition::SkipPolicyCheck, DefersLoadingPolicy::AllowDefersLoading, CachingPolicy::AllowCaching };
+    mainResourceLoadOptions.cache = toFetchOptionsCache(request.cachePolicy());
+
     CachedResourceRequest mainResourceRequest(ResourceRequest(request), mainResourceLoadOptions);
     if (!m_frame->isMainFrame() && m_frame->document()) {
         // If we are loading the main resource of a subframe, use the cache partition of the main document.

Source/WebCore/loader/FrameLoader.cpp

@@ -1193,10 +1193,12 @@ void FrameLoader::loadFrameRequest(FrameLoadRequest&& request, Event* event, For
         referrer = String();
 
     FrameLoadType loadType;
-    if (request.resourceRequest().cachePolicy() == ReloadIgnoringCacheData)
+    if (request.resourceRequest().cachePolicy() == RefreshAnyCacheData)
         loadType = FrameLoadType::Reload;
     else if (request.lockBackForwardList() == LockBackForwardList::Yes)
         loadType = FrameLoadType::RedirectWithLockedBackForwardList;
+    else if (request.resourceRequest().cachePolicy() == ReloadIgnoringCacheData)
+        loadType = FrameLoadType::ReloadFromOrigin;
     else
         loadType = FrameLoadType::Standard;
 
@@ -1281,7 +1283,7 @@ void FrameLoader::loadURL(FrameLoadRequest&& frameLoadRequest, const String& ref
 
     addExtraFieldsToRequest(request, newLoadType, true);
     if (isReload(newLoadType))
-        request.setCachePolicy(ReloadIgnoringCacheData);
+        request.setCachePolicy(RefreshAnyCacheData);
 
     ASSERT(newLoadType != FrameLoadType::Same);
 
@@ -1416,7 +1418,7 @@ void FrameLoader::load(DocumentLoader* newDocumentLoader)
     FrameLoadType type;
 
     if (shouldTreatURLAsSameAsCurrent(newDocumentLoader->originalRequest().url())) {
-        r.setCachePolicy(ReloadIgnoringCacheData);
+        r.setCachePolicy(RefreshAnyCacheData);
         type = FrameLoadType::Same;
     } else if (shouldTreatURLAsSameAsCurrent(newDocumentLoader->unreachableURL()) && m_loadType == FrameLoadType::Reload)
         type = FrameLoadType::Reload;
@@ -1641,8 +1643,7 @@ void FrameLoader::reload(OptionSet<ReloadOption> options)
 
     ResourceRequest& request = loader->request();
 
-    // FIXME: We don't have a mechanism to revalidate the main resource without reloading at the moment.
-    request.setCachePolicy(ReloadIgnoringCacheData);
+    request.setCachePolicy(RefreshAnyCacheData);
 
     // If we're about to re-post, set up action so the application can warn the user.
     if (request.httpMethod() == "POST")
@@ -2614,13 +2615,13 @@ ResourceRequestCachePolicy FrameLoader::defaultRequestCachingPolicy(const Resour
 
     if (isMainResource) {
         if (isReload(loadType) || request.isConditional())
-            return ReloadIgnoringCacheData;
+            return loadType == FrameLoadType::ReloadFromOrigin ? ReloadIgnoringCacheData : RefreshAnyCacheData;
 
         return UseProtocolCachePolicy;
     }
 
     if (request.isConditional())
-        return ReloadIgnoringCacheData;
+        return RefreshAnyCacheData;
 
     if (documentLoader()->isLoadingInAPISense()) {
         // If we inherit cache policy from a main resource, we use the DocumentLoader's
@@ -3478,8 +3479,10 @@ void FrameLoader::loadDifferentDocumentItem(HistoryItem& item, FrameLoadType loa
     } else {
         switch (loadType) {
         case FrameLoadType::Reload:
-        case FrameLoadType::ReloadFromOrigin:
         case FrameLoadType::ReloadExpiredOnly:
+            request.setCachePolicy(RefreshAnyCacheData);
+            break;
+        case FrameLoadType::ReloadFromOrigin:
             request.setCachePolicy(ReloadIgnoringCacheData);
             break;
         case FrameLoadType::Back:

Source/WebCore/loader/NavigationScheduler.cpp

@@ -182,7 +182,7 @@ class ScheduledRedirect : public ScheduledURLNavigation {
         UserGestureIndicator gestureIndicator { userGestureToForward() };
 
         bool refresh = equalIgnoringFragmentIdentifier(frame.document()->url(), url());
-        ResourceRequest resourceRequest { url(), referrer(), refresh ? ReloadIgnoringCacheData : UseProtocolCachePolicy };
+        ResourceRequest resourceRequest { url(), referrer(), refresh ? RefreshAnyCacheData : UseProtocolCachePolicy };
         FrameLoadRequest frameLoadRequest { initiatingDocument(), *securityOrigin(), resourceRequest, "_self", lockHistory(), lockBackForwardList(), MaybeSendReferrer, AllowNavigationToInvalidURL::No, NewFrameOpenerPolicy::Allow, shouldOpenExternalURLs(), initiatedByMainFrame() };
 
         frame.loader().changeLocation(WTFMove(frameLoadRequest));
@@ -216,7 +216,7 @@ class ScheduledRefresh : public ScheduledURLNavigation {
     {
         UserGestureIndicator gestureIndicator { userGestureToForward() };
 
-        ResourceRequest resourceRequest { url(), referrer(), ReloadIgnoringCacheData };
+        ResourceRequest resourceRequest { url(), referrer(), RefreshAnyCacheData };
         FrameLoadRequest frameLoadRequest { initiatingDocument(), *securityOrigin(), resourceRequest, "_self", lockHistory(), lockBackForwardList(), MaybeSendReferrer, AllowNavigationToInvalidURL::Yes, NewFrameOpenerPolicy::Allow, shouldOpenExternalURLs(), initiatedByMainFrame() };
 
         frame.loader().changeLocation(WTFMove(frameLoadRequest));
@@ -319,7 +319,7 @@ class ScheduledPageBlock final : public ScheduledNavigation {
         ResourceResponse replacementResponse { m_originDocument.url(), ASCIILiteral("text/plain"), 0, ASCIILiteral("UTF-8") };
         SubstituteData replacementData { SharedBuffer::create(), m_originDocument.url(), replacementResponse, SubstituteData::SessionHistoryVisibility::Hidden };
 
-        ResourceRequest resourceRequest { m_originDocument.url(), emptyString(), ReloadIgnoringCacheData };
+        ResourceRequest resourceRequest { m_originDocument.url(), emptyString(), RefreshAnyCacheData };
         FrameLoadRequest frameLoadRequest { m_originDocument, m_originDocument.securityOrigin(), resourceRequest, { }, lockHistory(), lockBackForwardList(), MaybeSendReferrer, AllowNavigationToInvalidURL::Yes, NewFrameOpenerPolicy::Allow, shouldOpenExternalURLs(), initiatedByMainFrame() };
         frameLoadRequest.setSubstituteData(replacementData);
         frame.loader().load(WTFMove(frameLoadRequest));