Skip to content
Permalink
Browse files
Implement listing origins for which CacheStorage is storing data
https://bugs.webkit.org/show_bug.cgi?id=178236

Patch by Youenn Fablet <youenn@apple.com> on 2017-10-13
Reviewed by Chris Dumez.

Source/WebKit:

Cache storage is split on per-origin folders which name is obfuscated through salting.
To retrieve the origin for each folder, an origin file is now stored within each folder.
This file contains the actual origin.

Adding support to get the list of origin by iterating through each folder and
getting the actual origin by reading the content of the 'origin' file.

Adding C API for WebKitTestRunner.

* NetworkProcess/cache/CacheStorageEngine.cpp:
(WebKit::CacheStorage::Engine::fetchEntries):
(WebKit::CacheStorage::ReadOriginsTaskCounter::create):
(WebKit::CacheStorage::ReadOriginsTaskCounter::~ReadOriginsTaskCounter):
(WebKit::CacheStorage::ReadOriginsTaskCounter::addOrigin):
(WebKit::CacheStorage::ReadOriginsTaskCounter::ReadOriginsTaskCounter):
* NetworkProcess/cache/CacheStorageEngine.h:
* NetworkProcess/cache/CacheStorageEngineCaches.cpp:
(WebKit::CacheStorage::cachesOriginFilename):
(WebKit::CacheStorage::Caches::retrieveOriginFromDirectory):
(WebKit::CacheStorage::Caches::Caches):
(WebKit::CacheStorage::Caches::storeOrigin):
(WebKit::CacheStorage::Caches::readOrigin):
(WebKit::CacheStorage::Caches::initialize):
* NetworkProcess/cache/CacheStorageEngineCaches.h:
(WebKit::CacheStorage::Caches::origin const):
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreGetFetchCacheOrigins):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:

Tools:

Adding hasDOMCache API for checking whether origin is storing data through Cache API.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::hasDOMCache):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::FetchCacheOriginsCallbackContext::FetchCacheOriginsCallbackContext):
(WTR::fetchCacheOriginsCallback):
(WTR::TestController::hasDOMCache):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

* http/tests/cache-storage/cache-clearing-origin.https.html:

Canonical link: https://commits.webkit.org/194510@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@223299 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
youennf authored and webkit-commit-queue committed Oct 13, 2017
1 parent f177212 commit 9112cf2c2ce26e8d10bf95e37caa2ee6020420c0
Showing 16 changed files with 286 additions and 18 deletions.
@@ -1,3 +1,12 @@
2017-10-13 Youenn Fablet <youenn@apple.com>

Implement listing origins for which CacheStorage is storing data
https://bugs.webkit.org/show_bug.cgi?id=178236

Reviewed by Chris Dumez.

* http/tests/cache-storage/cache-clearing-origin.https.html:

2017-10-13 Youenn Fablet <youenn@apple.com>

http/tests/cache-storage/cache-origins.https.html is flaky
@@ -22,12 +22,18 @@

if (!window.testRunner)
return Promise.reject("test runner needed");

assert_false(testRunner.hasDOMCache('https://localhost:80'), 'hasDOMCache with fake origin');
assert_true(testRunner.hasDOMCache(window.location.origin), "hasDOMCache with actual origin");

testRunner.clearDOMCache('https://localhost:80');

var keys = await self.caches.keys();
assert_not_equals(keys.length, 0, "keys should not be empty");

testRunner.clearDOMCache(window.location.origin);
assert_false(testRunner.hasDOMCache(window.location.origin), "Actual origin cache is cleared");

keys = await self.caches.keys();
assert_equals(keys.length, 0, "keys should be empty");

@@ -1,3 +1,39 @@
2017-10-13 Youenn Fablet <youenn@apple.com>

Implement listing origins for which CacheStorage is storing data
https://bugs.webkit.org/show_bug.cgi?id=178236

Reviewed by Chris Dumez.

Cache storage is split on per-origin folders which name is obfuscated through salting.
To retrieve the origin for each folder, an origin file is now stored within each folder.
This file contains the actual origin.

Adding support to get the list of origin by iterating through each folder and
getting the actual origin by reading the content of the 'origin' file.

Adding C API for WebKitTestRunner.

* NetworkProcess/cache/CacheStorageEngine.cpp:
(WebKit::CacheStorage::Engine::fetchEntries):
(WebKit::CacheStorage::ReadOriginsTaskCounter::create):
(WebKit::CacheStorage::ReadOriginsTaskCounter::~ReadOriginsTaskCounter):
(WebKit::CacheStorage::ReadOriginsTaskCounter::addOrigin):
(WebKit::CacheStorage::ReadOriginsTaskCounter::ReadOriginsTaskCounter):
* NetworkProcess/cache/CacheStorageEngine.h:
* NetworkProcess/cache/CacheStorageEngineCaches.cpp:
(WebKit::CacheStorage::cachesOriginFilename):
(WebKit::CacheStorage::Caches::retrieveOriginFromDirectory):
(WebKit::CacheStorage::Caches::Caches):
(WebKit::CacheStorage::Caches::storeOrigin):
(WebKit::CacheStorage::Caches::readOrigin):
(WebKit::CacheStorage::Caches::initialize):
* NetworkProcess/cache/CacheStorageEngineCaches.h:
(WebKit::CacheStorage::Caches::origin const):
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreGetFetchCacheOrigins):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:

2017-10-13 Alex Christensen <achristensen@webkit.org>

Fix API tests after r223269.
@@ -29,8 +29,9 @@
#include "NetworkCacheFileSystem.h"
#include "NetworkCacheIOChannel.h"
#include "NetworkProcess.h"
#include "WebsiteDataType.h"
#include <WebCore/CacheQueryOptions.h>
#include <WebCore/NotImplemented.h>
#include <WebCore/SecurityOrigin.h>
#include <pal/SessionID.h>
#include <wtf/CallbackAggregator.h>
#include <wtf/NeverDestroyed.h>
@@ -80,11 +81,9 @@ void Engine::destroyEngine(PAL::SessionID sessionID)
globalEngineMap().remove(sessionID);
}

void Engine::fetchEntries(PAL::SessionID sessionID, bool shouldComputeSize, WTF::Function<void(Vector<WebsiteData::Entry>)>&& completionHandler)
void Engine::fetchEntries(PAL::SessionID sessionID, bool shouldComputeSize, WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& completionHandler)
{
// FIXME: Support fetching entries
notImplemented();
completionHandler({ });
from(sessionID).fetchEntries(shouldComputeSize, WTFMove(completionHandler));
}

void Engine::clearAllEngines(WTF::Function<void()>&& completionHandler)
@@ -345,6 +344,54 @@ void Engine::removeCaches(const String& origin)
m_caches.remove(origin);
}

class ReadOriginsTaskCounter : public RefCounted<ReadOriginsTaskCounter> {
public:
static Ref<ReadOriginsTaskCounter> create(WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& callback)
{
return adoptRef(*new ReadOriginsTaskCounter(WTFMove(callback)));}

~ReadOriginsTaskCounter()
{
m_callback(WTFMove(m_entries));
}

void addOrigin(WebCore::SecurityOriginData&& origin)
{
m_entries.append(WebsiteData::Entry { WTFMove(origin), WebsiteDataType::DOMCache, 0 });
}

private:
explicit ReadOriginsTaskCounter(WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& callback)
: m_callback(WTFMove(callback))
{
}

WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)> m_callback;
Vector<WebsiteData::Entry> m_entries;
};

void Engine::fetchEntries(bool /* shouldComputeSize */, WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& completionHandler)
{
if (!shouldPersist()) {
auto entries = WTF::map(m_caches, [] (auto& pair) {
return WebsiteData::Entry { pair.value->origin(), WebsiteDataType::DOMCache, 0 };
});
completionHandler(WTFMove(entries));
return;
}

auto taskCounter = ReadOriginsTaskCounter::create(WTFMove(completionHandler));
for (auto& folderPath : WebCore::listDirectory(m_rootPath, "*")) {
if (!WebCore::fileIsDirectory(folderPath, WebCore::ShouldFollowSymbolicLinks::No))
continue;
Caches::retrieveOriginFromDirectory(folderPath, *m_ioQueue, [taskCounter = taskCounter.copyRef()] (std::optional<WebCore::SecurityOriginData>&& origin) mutable {
ASSERT(RunLoop::isMain());
if (origin)
taskCounter->addOrigin(WTFMove(origin.value()));
});
}
}

void Engine::clearAllCaches(CallbackAggregator& taskHandler)
{
for (auto& caches : m_caches.values())
@@ -59,7 +59,7 @@ class Engine : public ThreadSafeRefCounted<Engine> {
static void destroyEngine(PAL::SessionID);
static void clearAllEngines(WTF::Function<void()>&&);
static void clearEnginesForOrigins(const Vector<String>& origins, WTF::Function<void()>&&);
static void fetchEntries(PAL::SessionID, bool shouldComputeSize, WTF::Function<void(Vector<WebsiteData::Entry>)>&&);
static void fetchEntries(PAL::SessionID, bool shouldComputeSize, WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&&);

static Ref<Engine> create(String&& rootPath) { return adoptRef(*new Engine(WTFMove(rootPath))); }

@@ -95,6 +95,8 @@ class Engine : public ThreadSafeRefCounted<Engine> {

String cachesRootPath(const String& origin);

void fetchEntries(bool /* shouldComputeSize */, WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&&);

void initialize(WTF::Function<void(std::optional<WebCore::DOMCacheEngine::Error>&&)>&&);
void clearAllCaches(WTF::CallbackAggregator&);
void clearCachesForOrigin(const String& origin, WTF::CallbackAggregator&);
@@ -27,6 +27,8 @@
#include "CacheStorageEngine.h"

#include "NetworkCacheCoders.h"
#include "NetworkCacheIOChannel.h"
#include <WebCore/SecurityOrigin.h>
#include <wtf/RunLoop.h>
#include <wtf/UUID.h>
#include <wtf/text/StringBuilder.h>
@@ -43,14 +45,67 @@ static inline String cachesListFilename(const String& cachesRootPath)
return WebCore::pathByAppendingComponent(cachesRootPath, ASCIILiteral("cacheslist"));
}

static inline String cachesOriginFilename(const String& cachesRootPath)
{
return WebCore::pathByAppendingComponent(cachesRootPath, ASCIILiteral("origin"));
}

void Caches::retrieveOriginFromDirectory(const String& folderPath, WorkQueue& queue, WTF::CompletionHandler<void(std::optional<WebCore::SecurityOriginData>&&)>&& completionHandler)
{
queue.dispatch([completionHandler = WTFMove(completionHandler), folderPath = folderPath.isolatedCopy()]() mutable {
if (!WebCore::fileExists(cachesListFilename(folderPath))) {
RunLoop::main().dispatch([completionHandler = WTFMove(completionHandler)]() mutable {
completionHandler(std::nullopt);
});
return;
}

auto channel = IOChannel::open(cachesOriginFilename(folderPath), IOChannel::Type::Read);
channel->read(0, std::numeric_limits<size_t>::max(), nullptr, [completionHandler = WTFMove(completionHandler)](const Data& data, int error) mutable {
ASSERT(RunLoop::isMain());
if (error) {
completionHandler(std::nullopt);
return;
}
completionHandler(readOrigin(data));
});
});
}

Caches::Caches(Engine& engine, String&& origin, String&& rootPath, uint64_t quota)
: m_engine(&engine)
, m_origin(WTFMove(origin))
, m_origin(WebCore::SecurityOriginData::fromSecurityOrigin(WebCore::SecurityOrigin::createFromString(origin)))
, m_rootPath(WTFMove(rootPath))
, m_quota(quota)
{
}

void Caches::storeOrigin(CompletionCallback&& completionHandler)
{
WTF::Persistence::Encoder encoder;
encoder << m_origin.protocol;
encoder << m_origin.host;
encoder << m_origin.port;
m_engine->writeFile(cachesOriginFilename(m_rootPath), Data { encoder.buffer(), encoder.bufferSize() }, [protectedThis = makeRef(*this), completionHandler = WTFMove(completionHandler)] (std::optional<Error>&& error) mutable {
completionHandler(WTFMove(error));
});
}

std::optional<WebCore::SecurityOriginData> Caches::readOrigin(const Data& data)
{
// FIXME: We should be able to use modern decoders for persistent data.
WebCore::SecurityOriginData origin;
WTF::Persistence::Decoder decoder(data.data(), data.size());

if (!decoder.decode(origin.protocol))
return std::nullopt;
if (!decoder.decode(origin.host))
return std::nullopt;
if (!decoder.decode(origin.port))
return std::nullopt;
return WTFMove(origin);
}

void Caches::initialize(WebCore::DOMCacheEngine::CompletionCallback&& callback)
{
if (m_isInitialized) {
@@ -77,19 +132,28 @@ void Caches::initialize(WebCore::DOMCacheEngine::CompletionCallback&& callback)
}
m_storage = storage.releaseNonNull();
m_storage->writeWithoutWaiting();
readCachesFromDisk([this, callback = WTFMove(callback)](Expected<Vector<Cache>, Error>&& result) mutable {
makeDirty();

if (!result.hasValue()) {
callback(result.error());

auto pendingCallbacks = WTFMove(m_pendingInitializationCallbacks);
for (auto& callback : pendingCallbacks)
callback(result.error());
storeOrigin([this, callback = WTFMove(callback)] (std::optional<Error>&& error) mutable {
if (error) {
callback(Error::WriteDisk);
return;
}
m_caches = WTFMove(result.value());
initializeSize(WTFMove(callback));

readCachesFromDisk([this, callback = WTFMove(callback)](Expected<Vector<Cache>, Error>&& result) mutable {
makeDirty();

if (!result.hasValue()) {
callback(result.error());

auto pendingCallbacks = WTFMove(m_pendingInitializationCallbacks);
for (auto& callback : pendingCallbacks)
callback(result.error());
return;
}
m_caches = WTFMove(result.value());

initializeSize(WTFMove(callback));
});
});
}

@@ -27,6 +27,7 @@

#include "CacheStorageEngineCache.h"
#include "NetworkCacheStorage.h"
#include <WebCore/SecurityOriginData.h>
#include <wtf/CompletionHandler.h>

namespace WebKit {
@@ -39,6 +40,8 @@ class Caches : public RefCounted<Caches> {
public:
static Ref<Caches> create(Engine& engine, String&& origin, String&& rootPath, uint64_t quota) { return adoptRef(*new Caches { engine, WTFMove(origin), WTFMove(rootPath), quota }); }

static void retrieveOriginFromDirectory(const String& folderPath, WorkQueue&, WTF::CompletionHandler<void(std::optional<WebCore::SecurityOriginData>&&)>&&);

void initialize(WebCore::DOMCacheEngine::CompletionCallback&&);
void open(const String& name, WebCore::DOMCacheEngine::CacheIdentifierCallback&&);
void remove(uint64_t identifier, WebCore::DOMCacheEngine::CacheIdentifierCallback&&);
@@ -63,6 +66,7 @@ class Caches : public RefCounted<Caches> {
void removeRecord(const RecordInformation&);

const NetworkCache::Salt& salt() const;
const WebCore::SecurityOriginData& origin() const { return m_origin; }

bool shouldPersist() const { return !m_rootPath.isNull(); }

@@ -76,6 +80,9 @@ class Caches : public RefCounted<Caches> {
void readCachesFromDisk(WTF::Function<void(Expected<Vector<Cache>, WebCore::DOMCacheEngine::Error>&&)>&&);
void writeCachesToDisk(WebCore::DOMCacheEngine::CompletionCallback&&);

void storeOrigin(WebCore::DOMCacheEngine::CompletionCallback&&);
static std::optional<WebCore::SecurityOriginData> readOrigin(const NetworkCache::Data&);

Cache* find(const String& name);

void makeDirty() { ++m_updateCounter; }
@@ -84,7 +91,7 @@ class Caches : public RefCounted<Caches> {
bool m_isInitialized { false };
Engine* m_engine { nullptr };
uint64_t m_updateCounter { 0 };
String m_origin;
WebCore::SecurityOriginData m_origin;
String m_rootPath;
uint64_t m_quota { 0 };
uint64_t m_size { 0 };
@@ -26,6 +26,7 @@
#include "config.h"
#include "WKWebsiteDataStoreRef.h"

#include "APIArray.h"
#include "APIWebsiteDataStore.h"
#include "WKAPICast.h"
#include "WebResourceLoadStatisticsStore.h"
@@ -355,3 +356,15 @@ void WKWebsiteDataStoreRemoveAllServiceWorkerRegistrations(WKWebsiteDataStoreRef
UNUSED_PARAM(dataStoreRef);
#endif
}

void WKWebsiteDataStoreGetFetchCacheOrigins(WKWebsiteDataStoreRef dataStoreRef, void* context, WKWebsiteDataStoreGetFetchCacheOriginsFunction callback)
{
WebKit::toImpl(dataStoreRef)->websiteDataStore().fetchData(WebKit::WebsiteDataType::DOMCache, { }, [context, callback] (auto dataRecords) {
Vector<RefPtr<API::Object>> securityOrigins;
for (const auto& dataRecord : dataRecords) {
for (const auto& origin : dataRecord.origins)
securityOrigins.append(API::SecurityOrigin::create(origin.securityOrigin()));
}
callback(WebKit::toAPI(API::Array::create(WTFMove(securityOrigins)).ptr()), context);
});
}
@@ -78,6 +78,9 @@ WK_EXPORT void WKWebsiteDataStoreRemoveAllFetchCaches(WKWebsiteDataStoreRef data
WK_EXPORT void WKWebsiteDataStoreRemoveAllIndexedDatabases(WKWebsiteDataStoreRef dataStoreRef);
WK_EXPORT void WKWebsiteDataStoreRemoveAllServiceWorkerRegistrations(WKWebsiteDataStoreRef dataStoreRef);

typedef void (*WKWebsiteDataStoreGetFetchCacheOriginsFunction)(WKArrayRef, void*);
WK_EXPORT void WKWebsiteDataStoreGetFetchCacheOrigins(WKWebsiteDataStoreRef dataStoreRef, void* context, WKWebsiteDataStoreGetFetchCacheOriginsFunction function);

#ifdef __cplusplus
}
#endif
@@ -1,3 +1,24 @@
2017-10-13 Youenn Fablet <youenn@apple.com>

Implement listing origins for which CacheStorage is storing data
https://bugs.webkit.org/show_bug.cgi?id=178236

Reviewed by Chris Dumez.

Adding hasDOMCache API for checking whether origin is storing data through Cache API.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::hasDOMCache):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::FetchCacheOriginsCallbackContext::FetchCacheOriginsCallbackContext):
(WTR::fetchCacheOriginsCallback):
(WTR::TestController::hasDOMCache):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

2017-10-13 Alex Christensen <achristensen@webkit.org>

Remove Editor::simplifyMarkup
@@ -58,6 +58,7 @@ interface TestRunner {
void dumpPolicyDelegateCallbacks();

void clearDOMCache(DOMString origin);
boolean hasDOMCache(DOMString origin);

// Special options.
void keepWebHistory();

0 comments on commit 9112cf2

Please sign in to comment.