Skip to content
Permalink
Browse files
Use ResourceLoader to load appcache manifest
https://bugs.webkit.org/show_bug.cgi?id=182861

Patch by Youenn Fablet <youenn@apple.com> on 2018-02-21
Reviewed by Alex Christensen.

Source/WebCore:

Covered by updated tests.

Add ApplicationCacheResourceLoader to load an ApplicationCacheResource from a ResourceLoader.
Make use of it to load the app cache manifest.
Future work should load entries using the same loader.

Remove manifest handle.
Ensure that DocumentLoader does not register the manifest resource loader as its lifetime
is handled by its ApplicationCacheGroup.

Add a ResourceLoader option to bypass the application cache.
Use it for manifest loading.

* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* loader/DocumentLoader.cpp:
(WebCore::DocumentLoader::addSubresourceLoader):
* loader/ResourceLoaderOptions.h:
* loader/appcache/ApplicationCacheGroup.cpp:
(WebCore::ApplicationCacheGroup::stopLoading):
(WebCore::ApplicationCacheGroup::update):
(WebCore::ApplicationCacheGroup::createRequest):
(WebCore::ApplicationCacheGroup::didReceiveResponseAsync):
(WebCore::ApplicationCacheGroup::didReceiveData):
(WebCore::ApplicationCacheGroup::didFinishLoading):
(WebCore::ApplicationCacheGroup::didFail):
(WebCore::ApplicationCacheGroup::didFinishLoadingManifest):
(WebCore::ApplicationCacheGroup::checkIfLoadIsComplete):
* loader/appcache/ApplicationCacheGroup.h:
* loader/appcache/ApplicationCacheHost.cpp:
(WebCore::ApplicationCacheHost::maybeLoadResource):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForRedirect):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForResponse):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForError):
* loader/appcache/ApplicationCacheResourceLoader.cpp: Added.
* loader/appcache/ApplicationCacheResourceLoader.h: Added.

LayoutTests:

* http/tests/appcache/fail-on-update-2-expected.txt:
* http/tests/appcache/manifest-redirect-2-expected.txt:
* http/tests/appcache/offline-access-expected.txt:

Canonical link: https://commits.webkit.org/198767@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@228892 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
youennf authored and webkit-commit-queue committed Feb 21, 2018
1 parent ed4f541 commit 8d647566d7f5f0836bcf0e85518efd28a9335915
@@ -1,3 +1,14 @@
2018-02-21 Youenn Fablet <youenn@apple.com>

Use ResourceLoader to load appcache manifest
https://bugs.webkit.org/show_bug.cgi?id=182861

Reviewed by Alex Christensen.

* http/tests/appcache/fail-on-update-2-expected.txt:
* http/tests/appcache/manifest-redirect-2-expected.txt:
* http/tests/appcache/offline-access-expected.txt:

2018-02-21 Chris Dumez <cdumez@apple.com>

VTTCue constructor should use 'double' type for startTime / endTime
@@ -1,7 +1,7 @@
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
SUCCESS: No crash.
@@ -1,4 +1,5 @@
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because a redirection was attempted.
Test that a redirect makes resource caching fail.

Should say SUCCESS:
@@ -1,5 +1,6 @@
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
CONSOLE MESSAGE: Application Cache manifest could not be fetched, because a redirection was attempted.
CONSOLE MESSAGE: line 1: ApplicationCache is deprecated. Please use ServiceWorkers instead.
Test that offline applications work when there is no network access (simulated).

@@ -1,3 +1,47 @@
2018-02-21 Youenn Fablet <youenn@apple.com>

Use ResourceLoader to load appcache manifest
https://bugs.webkit.org/show_bug.cgi?id=182861

Reviewed by Alex Christensen.

Covered by updated tests.

Add ApplicationCacheResourceLoader to load an ApplicationCacheResource from a ResourceLoader.
Make use of it to load the app cache manifest.
Future work should load entries using the same loader.

Remove manifest handle.
Ensure that DocumentLoader does not register the manifest resource loader as its lifetime
is handled by its ApplicationCacheGroup.

Add a ResourceLoader option to bypass the application cache.
Use it for manifest loading.

* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* loader/DocumentLoader.cpp:
(WebCore::DocumentLoader::addSubresourceLoader):
* loader/ResourceLoaderOptions.h:
* loader/appcache/ApplicationCacheGroup.cpp:
(WebCore::ApplicationCacheGroup::stopLoading):
(WebCore::ApplicationCacheGroup::update):
(WebCore::ApplicationCacheGroup::createRequest):
(WebCore::ApplicationCacheGroup::didReceiveResponseAsync):
(WebCore::ApplicationCacheGroup::didReceiveData):
(WebCore::ApplicationCacheGroup::didFinishLoading):
(WebCore::ApplicationCacheGroup::didFail):
(WebCore::ApplicationCacheGroup::didFinishLoadingManifest):
(WebCore::ApplicationCacheGroup::checkIfLoadIsComplete):
* loader/appcache/ApplicationCacheGroup.h:
* loader/appcache/ApplicationCacheHost.cpp:
(WebCore::ApplicationCacheHost::maybeLoadResource):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForRedirect):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForResponse):
(WebCore::ApplicationCacheHost::maybeLoadFallbackForError):
* loader/appcache/ApplicationCacheResourceLoader.cpp: Added.
* loader/appcache/ApplicationCacheResourceLoader.h: Added.

2018-02-21 Don Olmstead <don.olmstead@sony.com>

[CMake][Win] Use cmakeconfig.h rather than config.h and Platform.h
@@ -1252,6 +1252,7 @@ loader/appcache/ApplicationCache.cpp
loader/appcache/ApplicationCacheGroup.cpp
loader/appcache/ApplicationCacheHost.cpp
loader/appcache/ApplicationCacheResource.cpp
loader/appcache/ApplicationCacheResourceLoader.cpp
loader/appcache/ApplicationCacheStorage.cpp
loader/appcache/DOMApplicationCache.cpp
loader/appcache/ManifestParser.cpp
@@ -7108,6 +7108,8 @@
418C395F1C8F0AAB0051C8A3 /* ReadableStreamDefaultController.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ReadableStreamDefaultController.h; sourceTree = "<group>"; };
418F88020FF957AE0080F045 /* JSAbstractWorker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSAbstractWorker.cpp; sourceTree = "<group>"; };
418F88030FF957AE0080F045 /* JSAbstractWorker.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = JSAbstractWorker.h; sourceTree = "<group>"; };
41945694203502A5004BA277 /* ApplicationCacheResourceLoader.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ApplicationCacheResourceLoader.cpp; sourceTree = "<group>"; };
41945696203502A6004BA277 /* ApplicationCacheResourceLoader.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ApplicationCacheResourceLoader.h; sourceTree = "<group>"; };
419ACF8E1F97E7D5009F1A83 /* ServiceWorkerFetch.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ServiceWorkerFetch.h; sourceTree = "<group>"; };
419ACF901F97E7D6009F1A83 /* ServiceWorkerFetch.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ServiceWorkerFetch.cpp; sourceTree = "<group>"; };
419BC2DC1685329900D64D6D /* VisitedLinkState.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = VisitedLinkState.cpp; sourceTree = "<group>"; };
@@ -15702,6 +15704,8 @@
24F54EAB101FE914000AE741 /* ApplicationCacheHost.h */,
1A8F6BB50DB55CDC001DB794 /* ApplicationCacheResource.cpp */,
1A8F6BB60DB55CDC001DB794 /* ApplicationCacheResource.h */,
41945694203502A5004BA277 /* ApplicationCacheResourceLoader.cpp */,
41945696203502A6004BA277 /* ApplicationCacheResourceLoader.h */,
1A2AAC560DC2A3B100A20D9A /* ApplicationCacheStorage.cpp */,
1A2AAC570DC2A3B100A20D9A /* ApplicationCacheStorage.h */,
1A8F6BB70DB55CDC001DB794 /* DOMApplicationCache.cpp */,
@@ -1543,6 +1543,10 @@ void DocumentLoader::addSubresourceLoader(ResourceLoader* loader)
ASSERT(!m_subresourceLoaders.contains(loader->identifier()));
ASSERT(!mainResourceLoader() || mainResourceLoader() != loader);

// Application Cache loaders are handled by their ApplicationCacheGroup directly.
if (loader->options().applicationCacheMode == ApplicationCacheMode::Bypass)
return;

// A page in the PageCache or about to enter PageCache should not be able to start loads.
ASSERT_WITH_SECURITY_IMPLICATION(!document() || document()->pageCacheState() == Document::NotInPageCache);

@@ -101,6 +101,11 @@ enum class ServiceWorkersMode {
Only // An error will happen if service worker is not handling the fetch. Used to bypass preflight safely.
};

enum class ApplicationCacheMode {
Use,
Bypass
};

enum class ContentEncodingSniffingPolicy {
Sniff,
DoNotSniff,
@@ -140,6 +145,7 @@ struct ResourceLoaderOptions : public FetchOptions {
SameOriginDataURLFlag sameOriginDataURLFlag { SameOriginDataURLFlag::Unset };
InitiatorContext initiatorContext { InitiatorContext::Document };
ServiceWorkersMode serviceWorkersMode { ServiceWorkersMode::All };
ApplicationCacheMode applicationCacheMode { ApplicationCacheMode::Use };
#if ENABLE(SERVICE_WORKER)
std::optional<ServiceWorkerRegistrationIdentifier> serviceWorkerRegistrationIdentifier;
// WebKit loading code is adding some HTTP headers between the application and the time service worker intercepts the fetch.
@@ -29,6 +29,7 @@
#include "ApplicationCache.h"
#include "ApplicationCacheHost.h"
#include "ApplicationCacheResource.h"
#include "ApplicationCacheResourceLoader.h"
#include "ApplicationCacheStorage.h"
#include "Chrome.h"
#include "ChromeClient.h"
@@ -314,18 +315,13 @@ void ApplicationCacheGroup::failedLoadingMainResource(DocumentLoader& loader)

void ApplicationCacheGroup::stopLoading()
{
if (m_manifestHandle) {
ASSERT(!m_currentHandle);

ASSERT(m_manifestHandle->client() == this);
m_manifestHandle->clearClient();

m_manifestHandle->cancel();
m_manifestHandle = nullptr;
if (m_loader) {
m_loader->cancel();
m_loader = nullptr;
}

if (m_currentHandle) {
ASSERT(!m_manifestHandle);
ASSERT(!m_loader);
ASSERT(m_cacheBeingUpdated);

ASSERT(m_currentHandle->client() == this);
@@ -436,14 +432,56 @@ void ApplicationCacheGroup::update(Frame& frame, ApplicationCacheUpdateOption up
postListenerTask(eventNames().checkingEvent, documentLoader);
}

ASSERT(!m_manifestHandle);
ASSERT(!m_loader);
ASSERT(!m_manifestResource);
ASSERT(!m_currentHandle);
ASSERT(!m_currentResource);
ASSERT(m_completionType == None);

// FIXME: Handle defer loading
m_manifestHandle = createResourceHandle(m_manifestURL, m_newestCache ? m_newestCache->manifestResource() : 0);

auto request = createRequest(URL { m_manifestURL }, m_newestCache ? m_newestCache->manifestResource() : nullptr);

m_currentResourceIdentifier = m_frame->page()->progress().createUniqueIdentifier();
InspectorInstrumentation::willSendRequest(m_frame, m_currentResourceIdentifier, m_frame->loader().documentLoader(), request, ResourceResponse { });

m_loader = ApplicationCacheResourceLoader::create(documentLoader.cachedResourceLoader(), WTFMove(request), [this] (auto&& resourceOrError) {
// 'this' is only valid if returned value is not Error::Abort.
if (!resourceOrError.has_value()) {
auto error = resourceOrError.error();
if (error == ApplicationCacheResourceLoader::Error::Abort)
return;
if (error == ApplicationCacheResourceLoader::Error::CannotCreateResource) {
// FIXME: We should get back the error from ApplicationCacheResourceLoader level.
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, ResourceError { ResourceError::Type::General });
this->cacheUpdateFailed();
return;
}
this->didFailLoadingManifest(error);
return;
}

m_manifestResource = WTFMove(resourceOrError.value());
this->didFinishLoadingManifest();
});
}

ResourceRequest ApplicationCacheGroup::createRequest(URL&& url, ApplicationCacheResource* resource)
{
ResourceRequest request { WTFMove(url) };
m_frame->loader().applyUserAgentIfNeeded(request);
request.setHTTPHeaderField(HTTPHeaderName::CacheControl, "max-age=0");

if (resource) {
const String& lastModified = resource->response().httpHeaderField(HTTPHeaderName::LastModified);
if (!lastModified.isEmpty())
request.setHTTPHeaderField(HTTPHeaderName::IfModifiedSince, lastModified);

const String& eTag = resource->response().httpHeaderField(HTTPHeaderName::ETag);
if (!eTag.isEmpty())
request.setHTTPHeaderField(HTTPHeaderName::IfNoneMatch, eTag);
}
return request;
}

void ApplicationCacheGroup::abort(Frame& frame)
@@ -494,12 +532,6 @@ void ApplicationCacheGroup::didReceiveResponseAsync(ResourceHandle* handle, Reso
ASSERT(m_frame);
InspectorInstrumentation::didReceiveResourceResponse(*m_frame, m_currentResourceIdentifier, m_frame->loader().documentLoader(), response, nullptr);

if (handle == m_manifestHandle) {
didReceiveManifestResponse(response);
completionHandler();
return;
}

ASSERT(handle == m_currentHandle);

URL url(handle->firstRequest().url());
@@ -578,16 +610,10 @@ void ApplicationCacheGroup::canAuthenticateAgainstProtectionSpaceAsync(ResourceH
void ApplicationCacheGroup::didReceiveData(ResourceHandle* handle, const char* data, unsigned length, int encodedDataLength)
{
UNUSED_PARAM(encodedDataLength);
ASSERT_UNUSED(handle, handle == m_currentHandle);

InspectorInstrumentation::didReceiveData(m_frame, m_currentResourceIdentifier, 0, length, 0);

if (handle == m_manifestHandle) {
didReceiveManifestData(data, length);
return;
}

ASSERT(handle == m_currentHandle);

ASSERT(m_currentResource);
m_currentResource->data().append(data, length);
}
@@ -598,11 +624,6 @@ void ApplicationCacheGroup::didFinishLoading(ResourceHandle* handle)
NetworkLoadMetrics emptyMetrics;
InspectorInstrumentation::didFinishLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, emptyMetrics, nullptr);

if (handle == m_manifestHandle) {
didFinishLoadingManifest();
return;
}

ASSERT(m_currentHandle == handle);
ASSERT(m_pendingEntries.contains(handle->firstRequest().url()));

@@ -633,12 +654,6 @@ void ApplicationCacheGroup::didFail(ResourceHandle* handle, const ResourceError&
{
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, error);

if (handle == m_manifestHandle) {
// A network error is logged elsewhere, no need to log again. Also, it's normal for manifest fetching to fail when working offline.
cacheUpdateFailed();
return;
}

ASSERT(handle == m_currentHandle);

unsigned type = m_currentResource ? m_currentResource->type() : m_pendingEntries.get(handle->firstRequest().url());
@@ -665,44 +680,6 @@ void ApplicationCacheGroup::didFail(ResourceHandle* handle, const ResourceError&
}
}

void ApplicationCacheGroup::didReceiveManifestResponse(const ResourceResponse& response)
{
ASSERT(!m_manifestResource);
ASSERT(m_manifestHandle);

if (response.httpStatusCode() == 404 || response.httpStatusCode() == 410) {
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_manifestHandle->firstRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(response.httpStatusCode()), " response."));
manifestNotFound();
return;
}

if (response.httpStatusCode() == 304)
return;

if (response.httpStatusCode() / 100 != 2) {
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_manifestHandle->firstRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(response.httpStatusCode()), " response."));
cacheUpdateFailed();
return;
}

if (response.url() != m_manifestHandle->firstRequest().url()) {
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_manifestHandle->firstRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, ASCIILiteral("Application Cache manifest could not be fetched, because a redirection was attempted."));
cacheUpdateFailed();
return;
}

m_manifestResource = ApplicationCacheResource::create(m_manifestHandle->firstRequest().url(), response, ApplicationCacheResource::Manifest);
}

void ApplicationCacheGroup::didReceiveManifestData(const char* data, int length)
{
if (m_manifestResource)
m_manifestResource->data().append(data, length);
}

void ApplicationCacheGroup::didFinishLoadingManifest()
{
bool isUpgradeAttempt = m_newestCache;
@@ -714,7 +691,7 @@ void ApplicationCacheGroup::didFinishLoadingManifest()
return;
}

m_manifestHandle = nullptr;
m_loader = nullptr;

// Check if the manifest was not modified.
if (isUpgradeAttempt) {
@@ -780,6 +757,36 @@ void ApplicationCacheGroup::didFinishLoadingManifest()
startLoadingEntry();
}

void ApplicationCacheGroup::didFailLoadingManifest(ApplicationCacheResourceLoader::Error error)
{
ASSERT(error != ApplicationCacheResourceLoader::Error::Abort && error != ApplicationCacheResourceLoader::Error::CannotCreateResource);

InspectorInstrumentation::didReceiveResourceResponse(*m_frame, m_currentResourceIdentifier, m_frame->loader().documentLoader(), m_loader->resource()->response(), nullptr);
switch (error) {
case ApplicationCacheResourceLoader::Error::NetworkError:
cacheUpdateFailed();
break;
case ApplicationCacheResourceLoader::Error::NotFound:
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_loader->resource()->resourceRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(m_loader->resource()->response().httpStatusCode()), " response."));
manifestNotFound();
break;
case ApplicationCacheResourceLoader::Error::NotOK:
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_loader->resource()->resourceRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because the manifest had a ", String::number(m_loader->resource()->response().httpStatusCode()), " response."));
cacheUpdateFailed();
break;
case ApplicationCacheResourceLoader::Error::RedirectForbidden:
InspectorInstrumentation::didFailLoading(m_frame, m_frame->loader().documentLoader(), m_currentResourceIdentifier, m_frame->loader().cancelledError(m_loader->resource()->resourceRequest()));
m_frame->document()->addConsoleMessage(MessageSource::AppCache, MessageLevel::Error, makeString("Application Cache manifest could not be fetched, because a redirection was attempted."));
cacheUpdateFailed();
break;
case ApplicationCacheResourceLoader::Error::CannotCreateResource:
case ApplicationCacheResourceLoader::Error::Abort:
break;
}
}

void ApplicationCacheGroup::didReachMaxAppCacheSize()
{
ASSERT(m_frame);
@@ -848,9 +855,9 @@ void ApplicationCacheGroup::manifestNotFound()

void ApplicationCacheGroup::checkIfLoadIsComplete()
{
if (m_manifestHandle || !m_pendingEntries.isEmpty() || m_downloadingPendingMasterResourceLoadersCount)
if (m_loader || !m_pendingEntries.isEmpty() || m_downloadingPendingMasterResourceLoadersCount)
return;

// We're done, all resources have finished downloading (successfully or not).

bool isUpgradeAttempt = m_newestCache;

0 comments on commit 8d64756

Please sign in to comment.