Skip to content

Commit

Permalink
Use ResourceLoader to load appcache manifest
Browse files Browse the repository at this point in the history
https://bugs.webkit.org/show_bug.cgi?id=182861

Patch by Youenn Fablet <youenn@apple.com> on 2018-02-21
Reviewed by Alex Christensen.

Source/WebCore:

Covered by updated tests.

Add ApplicationCacheResourceLoader to load an ApplicationCacheResource from a ResourceLoader.
Make use of it to load the app cache manifest.
Future work should load entries using the same loader.

Remove manifest handle.
Ensure that DocumentLoader does not register the manifest resource loader as its lifetime
is handled by its ApplicationCacheGroup.

Add a ResourceLoader option to bypass the application cache.
Use it for manifest loading.

* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* loader/DocumentLoader.cpp:
(WebCore::DocumentLoader::addSubresourceLoader):
* loader/ResourceLoaderOptions.h:
* loader/appcache/ApplicationCacheGroup.cpp:
(WebCore::ApplicationCacheGroup::stopLoading):
(WebCore::ApplicationCacheGroup::update):
(WebCore::ApplicationCacheGroup::createRequest):
(WebCore::ApplicationCacheGroup::didReceiveResponseAsync):
(WebCore::ApplicationCacheGroup::didReceiveData):
(WebCore::ApplicationCacheGroup::didFinishLoading):
(WebCore::ApplicationCacheGroup::didFail):
(WebCore::ApplicationCacheGroup::didFinishLoadingManifest):
(WebCore::ApplicationCacheGroup::checkIfLoadIsComplete):
* loader/appcache/ApplicationCacheGroup.h:
* loader/appcache/ApplicationCacheHost.cpp:
(WebCore::ApplicationCacheHost::maybeLoadResource):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForRedirect):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForResponse):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForError):
* loader/appcache/ApplicationCacheResourceLoader.cpp: Added.
* loader/appcache/ApplicationCacheResourceLoader.h: Added.

LayoutTests:

* http/tests/appcache/fail-on-update-2-expected.txt:
* http/tests/appcache/manifest-redirect-2-expected.txt:
* http/tests/appcache/offline-access-expected.txt:

Canonical link: https://commits.webkit.org/198767@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@228892 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
youennf authored and webkit-commit-queue committed Feb 21, 2018
1 parent ed4f541 commit 8d64756
Show file tree
Hide file tree
Showing 14 changed files with 377 additions and 81 deletions.
11 changes: 11 additions & 0 deletions LayoutTests/ChangeLog
@@ -1,3 +1,14 @@
2018-02-21 Youenn Fablet <youenn@apple.com>

Use ResourceLoader to load appcache manifest
https://bugs.webkit.org/show_bug.cgi?id=182861

Reviewed by Alex Christensen.

* http/tests/appcache/fail-on-update-2-expected.txt:
* http/tests/appcache/manifest-redirect-2-expected.txt:
* http/tests/appcache/offline-access-expected.txt:

2018-02-21 Chris Dumez <cdumez@apple.com>

VTTCue constructor should use 'double' type for startTime / endTime
Expand Down
@@ -1,7 +1,7 @@
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
SUCCESS: No crash.
@@ -1,4 +1,5 @@
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because a redirection was attempted.
Test that a redirect makes resource caching fail.

Should say SUCCESS:
Expand Down
@@ -1,5 +1,6 @@
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because a redirection was attempted.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
Test that offline applications work when there is no network access (simulated).

Expand Down
44 changes: 44 additions & 0 deletions Source/WebCore/ChangeLog
@@ -1,3 +1,47 @@
2018-02-21 Youenn Fablet <youenn@apple.com>

Use ResourceLoader to load appcache manifest
https://bugs.webkit.org/show_bug.cgi?id=182861

Reviewed by Alex Christensen.

Covered by updated tests.

Add ApplicationCacheResourceLoader to load an ApplicationCacheResource from a ResourceLoader.
Make use of it to load the app cache manifest.
Future work should load entries using the same loader.

Remove manifest handle.
Ensure that DocumentLoader does not register the manifest resource loader as its lifetime
is handled by its ApplicationCacheGroup.

Add a ResourceLoader option to bypass the application cache.
Use it for manifest loading.

* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* loader/DocumentLoader.cpp:
(WebCore::DocumentLoader::addSubresourceLoader):
* loader/ResourceLoaderOptions.h:
* loader/appcache/ApplicationCacheGroup.cpp:
(WebCore::ApplicationCacheGroup::stopLoading):
(WebCore::ApplicationCacheGroup::update):
(WebCore::ApplicationCacheGroup::createRequest):
(WebCore::ApplicationCacheGroup::didReceiveResponseAsync):
(WebCore::ApplicationCacheGroup::didReceiveData):
(WebCore::ApplicationCacheGroup::didFinishLoading):
(WebCore::ApplicationCacheGroup::didFail):
(WebCore::ApplicationCacheGroup::didFinishLoadingManifest):
(WebCore::ApplicationCacheGroup::checkIfLoadIsComplete):
* loader/appcache/ApplicationCacheGroup.h:
* loader/appcache/ApplicationCacheHost.cpp:
(WebCore::ApplicationCacheHost::maybeLoadResource):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForRedirect):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForResponse):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForError):
* loader/appcache/ApplicationCacheResourceLoader.cpp: Added.
* loader/appcache/ApplicationCacheResourceLoader.h: Added.

2018-02-21 Don Olmstead <don.olmstead@sony.com>

[CMake][Win] Use cmakeconfig.h rather than config.h and Platform.h
Expand Down
1 change: 1 addition & 0 deletions Source/WebCore/Sources.txt
Expand Up @@ -1252,6 +1252,7 @@ loader/appcache/ApplicationCache.cpp
loader/appcache/ApplicationCacheGroup.cpp
loader/appcache/ApplicationCacheHost.cpp
loader/appcache/ApplicationCacheResource.cpp
loader/appcache/ApplicationCacheResourceLoader.cpp
loader/appcache/ApplicationCacheStorage.cpp
loader/appcache/DOMApplicationCache.cpp
loader/appcache/ManifestParser.cpp
Expand Down
4 changes: 4 additions & 0 deletions Source/WebCore/WebCore.xcodeproj/project.pbxproj
Expand Up @@ -7108,6 +7108,8 @@
418C395F1C8F0AAB0051C8A3 /* ReadableStreamDefaultController.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ReadableStreamDefaultController.h; sourceTree = "<group>"; };
418F88020FF957AE0080F045 /* JSAbstractWorker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSAbstractWorker.cpp; sourceTree = "<group>"; };
418F88030FF957AE0080F045 /* JSAbstractWorker.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSAbstractWorker.h; sourceTree = "<group>"; };
41945694203502A5004BA277 /* ApplicationCacheResourceLoader.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ApplicationCacheResourceLoader.cpp; sourceTree = "<group>"; };
41945696203502A6004BA277 /* ApplicationCacheResourceLoader.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ApplicationCacheResourceLoader.h; sourceTree = "<group>"; };
419ACF8E1F97E7D5009F1A83 /* ServiceWorkerFetch.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ServiceWorkerFetch.h; sourceTree = "<group>"; };
419ACF901F97E7D6009F1A83 /* ServiceWorkerFetch.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ServiceWorkerFetch.cpp; sourceTree = "<group>"; };
419BC2DC1685329900D64D6D /* VisitedLinkState.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = VisitedLinkState.cpp; sourceTree = "<group>"; };
Expand Down Expand Up @@ -15702,6 +15704,8 @@
24F54EAB101FE914000AE741 /* ApplicationCacheHost.h */,
1A8F6BB50DB55CDC001DB794 /* ApplicationCacheResource.cpp */,
1A8F6BB60DB55CDC001DB794 /* ApplicationCacheResource.h */,
41945694203502A5004BA277 /* ApplicationCacheResourceLoader.cpp */,
41945696203502A6004BA277 /* ApplicationCacheResourceLoader.h */,
1A2AAC560DC2A3B100A20D9A /* ApplicationCacheStorage.cpp */,
1A2AAC570DC2A3B100A20D9A /* ApplicationCacheStorage.h */,
1A8F6BB70DB55CDC001DB794 /* DOMApplicationCache.cpp */,
Expand Down
4 changes: 4 additions & 0 deletions Source/WebCore/loader/DocumentLoader.cpp
Expand Up @@ -1543,6 +1543,10 @@ void DocumentLoader::addSubresourceLoader(ResourceLoader* loader)
ASSERT(!m_subresourceLoaders.contains(loader->identifier()));
ASSERT(!mainResourceLoader() || mainResourceLoader() != loader);

// Application Cache loaders are handled by their ApplicationCacheGroup directly.
if (loader->options().applicationCacheMode == ApplicationCacheMode::Bypass)
return;

// A page in the PageCache or about to enter PageCache should not be able to start loads.
ASSERT_WITH_SECURITY_IMPLICATION(!document() || document()->pageCacheState() == Document::NotInPageCache);

Expand Down
6 changes: 6 additions & 0 deletions Source/WebCore/loader/ResourceLoaderOptions.h
Expand Up @@ -101,6 +101,11 @@ enum class ServiceWorkersMode {
Only // An error will happen if service worker is not handling the fetch. Used to bypass preflight safely.
};

enum class ApplicationCacheMode {
Use,
Bypass
};

enum class ContentEncodingSniffingPolicy {
Sniff,
DoNotSniff,
Expand Down Expand Up @@ -140,6 +145,7 @@ struct ResourceLoaderOptions : public FetchOptions {
SameOriginDataURLFlag sameOriginDataURLFlag { SameOriginDataURLFlag::Unset };
InitiatorContext initiatorContext { InitiatorContext::Document };
ServiceWorkersMode serviceWorkersMode { ServiceWorkersMode::All };
ApplicationCacheMode applicationCacheMode { ApplicationCacheMode::Use };
#if ENABLE(SERVICE_WORKER)
std::optional<ServiceWorkerRegistrationIdentifier> serviceWorkerRegistrationIdentifier;
// WebKit loading code is adding some HTTP headers between the application and the time service worker intercepts the fetch.
Expand Down
161 changes: 84 additions & 77 deletions Source/WebCore/loader/appcache/ApplicationCacheGroup.cpp
Expand Up @@ -29,6 +29,7 @@
#include "ApplicationCache.h"
#include "ApplicationCacheHost.h"
#include "ApplicationCacheResource.h"
#include "ApplicationCacheResourceLoader.h"
#include "ApplicationCacheStorage.h"
#include "Chrome.h"
#include "ChromeClient.h"
Expand Down Expand Up @@ -314,18 +315,13 @@ void ApplicationCacheGroup::failedLoadingMainResource(DocumentLoader& loader)

void ApplicationCacheGroup::stopLoading()
{
if (m_manifestHandle) {
ASSERT(!m_currentHandle);

ASSERT(m_manifestHandle->client() == this);
m_manifestHandle->clearClient();

m_manifestHandle->cancel();
m_manifestHandle = nullptr;
if (m_loader) {
m_loader->cancel();
m_loader = nullptr;
}

if (m_currentHandle) {
ASSERT(!m_manifestHandle);
ASSERT(!m_loader);
ASSERT(m_cacheBeingUpdated);

ASSERT(m_currentHandle->client() == this);
Expand Down Expand Up @@ -436,14 +432,56 @@ void ApplicationCacheGroup::update(Frame& frame, ApplicationCacheUpdateOption up
postListenerTask(eventNames().checkingEvent, documentLoader);
}

ASSERT(!m_manifestHandle);
ASSERT(!m_loader);
ASSERT(!m_manifestResource);
ASSERT(!m_currentHandle);
ASSERT(!m_currentResource);
ASSERT(m_completionType == None);

// FIXME: Handle defer loading
m_manifestHandle = createResourceHandle(m_manifestURL, m_newestCache ? m_newestCache->manifestResource() : 0);

auto request = createRequest(URL { m_manifestURL }, m_newestCache ? m_newestCache->manifestResource() : nullptr);

m_currentResourceIdentifier = m_frame->page()->progress().createUniqueIdentifier();
InspectorInstrumentation::willSendRequest(m_frame, m_currentResourceIdentifier, m_frame->loader().documentLoader(), request, ResourceResponse { });

m_loader = ApplicationCacheResourceLoader::create(documentLoader.cachedResourceLoader(), WTFMove(request), [this] (auto&& resourceOrError) {
// 'this' is only valid if returned value is not Error::Abort.
if (!resourceOrError.has_value()) {
auto error = resourceOrError.error();
if (error == ApplicationCacheResourceLoader::Error::Abort)
return;
if (error == ApplicationCacheResourceLoader::Error::CannotCreateResource) {
// FIXME: We should get back the error from ApplicationCacheResourceLoader level.
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, ResourceError { ResourceError::Type::General });
this->cacheUpdateFailed();
return;
}
this->didFailLoadingManifest(error);
return;
}

m_manifestResource = WTFMove(resourceOrError.value());
this->didFinishLoadingManifest();
});
}

ResourceRequest ApplicationCacheGroup::createRequest(URL&& url, ApplicationCacheResource* resource)
{
ResourceRequest request { WTFMove(url) };
m_frame->loader().applyUserAgentIfNeeded(request);
request.setHTTPHeaderField(HTTPHeaderName::CacheControl, "max-age=0");

if (resource) {
const String& lastModified = resource->response().httpHeaderField(HTTPHeaderName::LastModified);
if (!lastModified.isEmpty())
request.setHTTPHeaderField(HTTPHeaderName::IfModifiedSince, lastModified);

const String& eTag = resource->response().httpHeaderField(HTTPHeaderName::ETag);
if (!eTag.isEmpty())
request.setHTTPHeaderField(HTTPHeaderName::IfNoneMatch, eTag);
}
return request;
}

void ApplicationCacheGroup::abort(Frame& frame)
Expand Down Expand Up @@ -494,12 +532,6 @@ void ApplicationCacheGroup::didReceiveResponseAsync(ResourceHandle* handle, Reso
ASSERT(m_frame);
InspectorInstrumentation::didReceiveResourceResponse(*m_frame, m_currentResourceIdentifier, m_frame->loader().documentLoader(), response, nullptr);

if (handle == m_manifestHandle) {
didReceiveManifestResponse(response);
completionHandler();
return;
}

ASSERT(handle == m_currentHandle);

URL url(handle->firstRequest().url());
Expand Down Expand Up @@ -578,16 +610,10 @@ void ApplicationCacheGroup::canAuthenticateAgainstProtectionSpaceAsync(ResourceH
void ApplicationCacheGroup::didReceiveData(ResourceHandle* handle, const char* data, unsigned length, int encodedDataLength)
{
UNUSED_PARAM(encodedDataLength);
ASSERT_UNUSED(handle, handle == m_currentHandle);

InspectorInstrumentation::didReceiveData(m_frame, m_currentResourceIdentifier, 0, length, 0);

if (handle == m_manifestHandle) {
didReceiveManifestData(data, length);
return;
}

ASSERT(handle == m_currentHandle);

ASSERT(m_currentResource);
m_currentResource->data().append(data, length);
}
Expand All @@ -598,11 +624,6 @@ void ApplicationCacheGroup::didFinishLoading(ResourceHandle* handle)
NetworkLoadMetrics emptyMetrics;
InspectorInstrumentation::didFinishLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, emptyMetrics, nullptr);

if (handle == m_manifestHandle) {
didFinishLoadingManifest();
return;
}

ASSERT(m_currentHandle == handle);
ASSERT(m_pendingEntries.contains(handle->firstRequest().url()));

Expand Down Expand Up @@ -633,12 +654,6 @@ void ApplicationCacheGroup::didFail(ResourceHandle* handle, const ResourceError&
{
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, error);

if (handle == m_manifestHandle) {
// A network error is logged elsewhere, no need to log again. Also, it's normal for manifest fetching to fail when working offline.
cacheUpdateFailed();
return;
}

ASSERT(handle == m_currentHandle);

unsigned type = m_currentResource ? m_currentResource->type() : m_pendingEntries.get(handle->firstRequest().url());
Expand All @@ -665,44 +680,6 @@ void ApplicationCacheGroup::didFail(ResourceHandle* handle, const ResourceError&
}
}

void ApplicationCacheGroup::didReceiveManifestResponse(const ResourceResponse& response)
{
ASSERT(!m_manifestResource);
ASSERT(m_manifestHandle);

if (response.httpStatusCode() == 404 || response.httpStatusCode() == 410) {
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_manifestHandle->firstRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(response.httpStatusCode()), " response."));
manifestNotFound();
return;
}

if (response.httpStatusCode() == 304)
return;

if (response.httpStatusCode() / 100 != 2) {
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_manifestHandle->firstRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(response.httpStatusCode()), " response."));
cacheUpdateFailed();
return;
}

if (response.url() != m_manifestHandle->firstRequest().url()) {
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_manifestHandle->firstRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, ASCIILiteral("Application Cache manifest could not be fetched, because a redirection was attempted."));
cacheUpdateFailed();
return;
}

m_manifestResource = ApplicationCacheResource::create(m_manifestHandle->firstRequest().url(), response, ApplicationCacheResource::Manifest);
}

void ApplicationCacheGroup::didReceiveManifestData(const char* data, int length)
{
if (m_manifestResource)
m_manifestResource->data().append(data, length);
}

void ApplicationCacheGroup::didFinishLoadingManifest()
{
bool isUpgradeAttempt = m_newestCache;
Expand All @@ -714,7 +691,7 @@ void ApplicationCacheGroup::didFinishLoadingManifest()
return;
}

m_manifestHandle = nullptr;
m_loader = nullptr;

// Check if the manifest was not modified.
if (isUpgradeAttempt) {
Expand Down Expand Up @@ -780,6 +757,36 @@ void ApplicationCacheGroup::didFinishLoadingManifest()
startLoadingEntry();
}

void ApplicationCacheGroup::didFailLoadingManifest(ApplicationCacheResourceLoader::Error error)
{
ASSERT(error != ApplicationCacheResourceLoader::Error::Abort && error != ApplicationCacheResourceLoader::Error::CannotCreateResource);

InspectorInstrumentation::didReceiveResourceResponse(*m_frame, m_currentResourceIdentifier, m_frame->loader().documentLoader(), m_loader->resource()->response(), nullptr);
switch (error) {
case ApplicationCacheResourceLoader::Error::NetworkError:
cacheUpdateFailed();
break;
case ApplicationCacheResourceLoader::Error::NotFound:
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_loader->resource()->resourceRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(m_loader->resource()->response().httpStatusCode()), " response."));
manifestNotFound();
break;
case ApplicationCacheResourceLoader::Error::NotOK:
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_loader->resource()->resourceRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(m_loader->resource()->response().httpStatusCode()), " response."));
cacheUpdateFailed();
break;
case ApplicationCacheResourceLoader::Error::RedirectForbidden:
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_loader->resource()->resourceRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because a redirection was attempted."));
cacheUpdateFailed();
break;
case ApplicationCacheResourceLoader::Error::CannotCreateResource:
case ApplicationCacheResourceLoader::Error::Abort:
break;
}
}

void ApplicationCacheGroup::didReachMaxAppCacheSize()
{
ASSERT(m_frame);
Expand Down Expand Up @@ -848,9 +855,9 @@ void ApplicationCacheGroup::manifestNotFound()

void ApplicationCacheGroup::checkIfLoadIsComplete()
{
if (m_manifestHandle || !m_pendingEntries.isEmpty() || m_downloadingPendingMasterResourceLoadersCount)
if (m_loader || !m_pendingEntries.isEmpty() || m_downloadingPendingMasterResourceLoadersCount)
return;

// We're done, all resources have finished downloading (successfully or not).

bool isUpgradeAttempt = m_newestCache;
Expand Down

0 comments on commit 8d64756

Please sign in to comment.