Skip to content
Permalink
Browse files
Cache API implementation should be able to compute storage size for W…
…ebKit client applications.

https://bugs.webkit.org/show_bug.cgi?id=178350

Patch by Youenn Fablet <youenn@apple.com> on 2017-10-17
Reviewed by Chris Dumez.

Source/WebCore:

* page/SecurityOriginData.h:
(WebCore::SecurityOriginData::equals const):

Source/WebKit:

When gathering data from DOM Cache, we compute the size as follows:
- If Caches object is not persistent, size is zero
- If Caches object is persistent, we use the size computed by NetworkCache::Storage.
Covered by updated tests.

* NetworkProcess/cache/CacheStorageEngine.cpp:
(WebKit::CacheStorage::ReadOriginsTaskCounter::create):
(WebKit::CacheStorage::ReadOriginsTaskCounter::addOrigin):
(WebKit::CacheStorage::Engine::fetchEntries):
* NetworkProcess/cache/CacheStorageEngineCaches.cpp:
(WebKit::CacheStorage::Caches::storageSize const):
* NetworkProcess/cache/CacheStorageEngineCaches.h:
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreGetFetchCacheSizeForOrigin):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:

Tools:

Adding support for a domCacheSize getter.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::domCacheSize):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::FetchCacheSizeForOriginCallbackContext::FetchCacheSizeForOriginCallbackContext):
(WTR::fetchCacheSizeForOriginCallback):
(WTR::TestController::domCacheSize):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

LayoutTests:

* http/tests/cache-storage/cache-clearing-origin.https.html:
* http/tests/cache-storage/cache-representation.https.html:

Canonical link: https://commits.webkit.org/194620@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@223558 268f45cc-cd09-0410-ab3c-d52691b4dbfc
  • Loading branch information
youennf authored and webkit-commit-queue committed Oct 17, 2017
1 parent 64793d5 commit 7ea502a055b97b40cc7b3525f49abda8ee93bcf3
@@ -1,3 +1,13 @@
2017-10-17 Youenn Fablet <youenn@apple.com>

Cache API implementation should be able to compute storage size for WebKit client applications.
https://bugs.webkit.org/show_bug.cgi?id=178350

Reviewed by Chris Dumez.

* http/tests/cache-storage/cache-clearing-origin.https.html:
* http/tests/cache-storage/cache-representation.https.html:

2017-10-17 Ryan Haddad <ryanhaddad@apple.com>

Add expected result file for fast/css/DOMQuad-points-SameObject.html.
@@ -30,9 +30,11 @@

var keys = await self.caches.keys();
assert_not_equals(keys.length, 0, "keys should not be empty");
assert_true(testRunner.domCacheSize(window.location.origin) > 0, "Actual origin cache size is not zero");

testRunner.clearDOMCache(window.location.origin);
assert_false(testRunner.hasDOMCache(window.location.origin), "Actual origin cache is cleared");
assert_equals(testRunner.domCacheSize(window.location.origin), 0, "Actual origin cache size is zero");

keys = await self.caches.keys();
assert_equals(keys.length, 0, "keys should be empty");
@@ -18,6 +18,10 @@
var caches = results[0].caches;
assert_equals(!!caches["persistent"].length, hasPersistent, "persistent");
assert_equals(!!caches["removed"].length, hasRemoved, "removed");

if (!hasPersistent)
assert_equals(testRunner.domCacheSize(results.origin), 0, "non persistent cache storage size is zero");

}, name);
}

@@ -1,3 +1,13 @@
2017-10-17 Youenn Fablet <youenn@apple.com>

Cache API implementation should be able to compute storage size for WebKit client applications.
https://bugs.webkit.org/show_bug.cgi?id=178350

Reviewed by Chris Dumez.

* page/SecurityOriginData.h:
(WebCore::SecurityOriginData::equals const):

2017-10-17 Daniel Bates <dabates@apple.com>

ASSERTION FAILED: m_truncation != cFullTruncation in InlineTextBox::clampedOffset()
@@ -1,3 +1,26 @@
2017-10-17 Youenn Fablet <youenn@apple.com>

Cache API implementation should be able to compute storage size for WebKit client applications.
https://bugs.webkit.org/show_bug.cgi?id=178350

Reviewed by Chris Dumez.

When gathering data from DOM Cache, we compute the size as follows:
- If Caches object is not persistent, size is zero
- If Caches object is persistent, we use the size computed by NetworkCache::Storage.
Covered by updated tests.

* NetworkProcess/cache/CacheStorageEngine.cpp:
(WebKit::CacheStorage::ReadOriginsTaskCounter::create):
(WebKit::CacheStorage::ReadOriginsTaskCounter::addOrigin):
(WebKit::CacheStorage::Engine::fetchEntries):
* NetworkProcess/cache/CacheStorageEngineCaches.cpp:
(WebKit::CacheStorage::Caches::storageSize const):
* NetworkProcess/cache/CacheStorageEngineCaches.h:
* UIProcess/API/C/WKWebsiteDataStoreRef.cpp:
(WKWebsiteDataStoreGetFetchCacheSizeForOrigin):
* UIProcess/API/C/WKWebsiteDataStoreRef.h:

2017-10-17 Keith Miller <keith_miller@apple.com>

Change WebCore sources to work with unified source builds
@@ -348,16 +348,17 @@ class ReadOriginsTaskCounter : public RefCounted<ReadOriginsTaskCounter> {
public:
static Ref<ReadOriginsTaskCounter> create(WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& callback)
{
return adoptRef(*new ReadOriginsTaskCounter(WTFMove(callback)));}
return adoptRef(*new ReadOriginsTaskCounter(WTFMove(callback)));
}

~ReadOriginsTaskCounter()
{
m_callback(WTFMove(m_entries));
}

void addOrigin(WebCore::SecurityOriginData&& origin)
void addOrigin(WebCore::SecurityOriginData&& origin, uint64_t size)
{
m_entries.append(WebsiteData::Entry { WTFMove(origin), WebsiteDataType::DOMCache, 0 });
m_entries.append(WebsiteData::Entry { WTFMove(origin), WebsiteDataType::DOMCache, size });
}

private:
@@ -370,7 +371,7 @@ class ReadOriginsTaskCounter : public RefCounted<ReadOriginsTaskCounter> {
Vector<WebsiteData::Entry> m_entries;
};

void Engine::fetchEntries(bool /* shouldComputeSize */, WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& completionHandler)
void Engine::fetchEntries(bool shouldComputeSize, WTF::CompletionHandler<void(Vector<WebsiteData::Entry>)>&& completionHandler)
{
if (!shouldPersist()) {
auto entries = WTF::map(m_caches, [] (auto& pair) {
@@ -384,10 +385,23 @@ void Engine::fetchEntries(bool /* shouldComputeSize */, WTF::CompletionHandler<v
for (auto& folderPath : WebCore::listDirectory(m_rootPath, "*")) {
if (!WebCore::fileIsDirectory(folderPath, WebCore::ShouldFollowSymbolicLinks::No))
continue;
Caches::retrieveOriginFromDirectory(folderPath, *m_ioQueue, [taskCounter = taskCounter.copyRef()] (std::optional<WebCore::SecurityOriginData>&& origin) mutable {
Caches::retrieveOriginFromDirectory(folderPath, *m_ioQueue, [protectedThis = makeRef(*this), shouldComputeSize, taskCounter = taskCounter.copyRef()] (std::optional<WebCore::SecurityOriginData>&& origin) mutable {
ASSERT(RunLoop::isMain());
if (origin)
taskCounter->addOrigin(WTFMove(origin.value()));
if (!origin)
return;

if (!shouldComputeSize) {
taskCounter->addOrigin(WTFMove(origin.value()), 0);
return;
}

auto cacheOrigin = origin->securityOrigin()->toString();
protectedThis->readCachesFromDisk(cacheOrigin, [origin = WTFMove(origin.value()), taskCounter = WTFMove(taskCounter)] (CachesOrError&& result) mutable {
if (!result.hasValue())
return;
taskCounter->addOrigin(WTFMove(origin), result.value().get().storageSize());

});
});
}
}
@@ -530,6 +530,14 @@ void Caches::appendRepresentation(StringBuilder& builder) const
builder.append("]}\n");
}

uint64_t Caches::storageSize() const
{
ASSERT(m_isInitialized);
if (!shouldPersist())
return 0;
return m_storage->approximateSize();
}

} // namespace CacheStorage

} // namespace WebKit
@@ -73,6 +73,8 @@ class Caches : public RefCounted<Caches> {
void clear(WTF::CompletionHandler<void()>&&);
void clearMemoryRepresentation();

uint64_t storageSize() const;

private:
Caches(Engine&, String&& origin, String&& rootPath, uint64_t quota);

@@ -29,9 +29,11 @@
#include "APIArray.h"
#include "APIWebsiteDataStore.h"
#include "WKAPICast.h"
#include "WKSecurityOriginRef.h"
#include "WebResourceLoadStatisticsStore.h"
#include "WebResourceLoadStatisticsTelemetry.h"
#include "WebsiteData.h"
#include "WebsiteDataFetchOption.h"
#include "WebsiteDataRecord.h"
#include "WebsiteDataType.h"
#include <WebCore/URL.h>
@@ -368,3 +370,22 @@ void WKWebsiteDataStoreGetFetchCacheOrigins(WKWebsiteDataStoreRef dataStoreRef,
callback(WebKit::toAPI(API::Array::create(WTFMove(securityOrigins)).ptr()), context);
});
}

void WKWebsiteDataStoreGetFetchCacheSizeForOrigin(WKWebsiteDataStoreRef dataStoreRef, WKStringRef origin, void* context, WKWebsiteDataStoreGetFetchCacheSizeForOriginFunction callback)
{
OptionSet<WebKit::WebsiteDataFetchOption> fetchOptions = WebKit::WebsiteDataFetchOption::ComputeSizes;

WebKit::toImpl(dataStoreRef)->websiteDataStore().fetchData(WebKit::WebsiteDataType::DOMCache, fetchOptions, [origin, context, callback] (auto dataRecords) {
auto originData = WebCore::SecurityOriginData::fromSecurityOrigin(WebCore::SecurityOrigin::createFromString(WebKit::toImpl(origin)->string()));
for (auto& dataRecord : dataRecords) {
for (const auto& recordOrigin : dataRecord.origins) {
if (originData == recordOrigin) {
callback(dataRecord.size ? dataRecord.size->totalSize : 0, context);
return;
}

}
}
callback(0, context);
});
}
@@ -81,6 +81,9 @@ WK_EXPORT void WKWebsiteDataStoreRemoveAllServiceWorkerRegistrations(WKWebsiteDa
typedef void (*WKWebsiteDataStoreGetFetchCacheOriginsFunction)(WKArrayRef, void*);
WK_EXPORT void WKWebsiteDataStoreGetFetchCacheOrigins(WKWebsiteDataStoreRef dataStoreRef, void* context, WKWebsiteDataStoreGetFetchCacheOriginsFunction function);

typedef void (*WKWebsiteDataStoreGetFetchCacheSizeForOriginFunction)(uint64_t, void*);
WK_EXPORT void WKWebsiteDataStoreGetFetchCacheSizeForOrigin(WKWebsiteDataStoreRef dataStoreRef, WKStringRef origin, void* context, WKWebsiteDataStoreGetFetchCacheSizeForOriginFunction function);

#ifdef __cplusplus
}
#endif
@@ -1,3 +1,24 @@
2017-10-17 Youenn Fablet <youenn@apple.com>

Cache API implementation should be able to compute storage size for WebKit client applications.
https://bugs.webkit.org/show_bug.cgi?id=178350

Reviewed by Chris Dumez.

Adding support for a domCacheSize getter.

* WebKitTestRunner/InjectedBundle/Bindings/TestRunner.idl:
* WebKitTestRunner/InjectedBundle/TestRunner.cpp:
(WTR::TestRunner::domCacheSize):
* WebKitTestRunner/InjectedBundle/TestRunner.h:
* WebKitTestRunner/TestController.cpp:
(WTR::FetchCacheSizeForOriginCallbackContext::FetchCacheSizeForOriginCallbackContext):
(WTR::fetchCacheSizeForOriginCallback):
(WTR::TestController::domCacheSize):
* WebKitTestRunner/TestController.h:
* WebKitTestRunner/TestInvocation.cpp:
(WTR::TestInvocation::didReceiveSynchronousMessageFromInjectedBundle):

2017-10-17 Nael Ouedraogo <nael.ouedraogo@crf.canon.fr>

[GStreamer][GTK][WPE] update-webkit-libs-jhbuild fails to detect changes in included moduleset files
@@ -59,6 +59,7 @@ interface TestRunner {

void clearDOMCache(DOMString origin);
boolean hasDOMCache(DOMString origin);
unsigned long domCacheSize(DOMString origin);

// Special options.
void keepWebHistory();
@@ -1838,4 +1838,13 @@ bool TestRunner::hasDOMCache(JSStringRef origin)
return WKBooleanGetValue(static_cast<WKBooleanRef>(returnData));
}

uint64_t TestRunner::domCacheSize(JSStringRef origin)
{
WKRetainPtr<WKStringRef> messageName(AdoptWK, WKStringCreateWithUTF8CString("DOMCacheSize"));
WKRetainPtr<WKStringRef> messageBody(AdoptWK, WKStringCreateWithJSString(origin));
WKTypeRef returnData = 0;
WKBundlePagePostSynchronousMessageForTesting(InjectedBundle::singleton().page()->page(), messageName.get(), messageBody.get(), &returnData);
return WKUInt64GetValue(static_cast<WKUInt64Ref>(returnData));
}

} // namespace WTR
@@ -165,6 +165,7 @@ class TestRunner : public JSWrappable {

void clearDOMCache(JSStringRef origin);
bool hasDOMCache(JSStringRef origin);
uint64_t domCacheSize(JSStringRef origin);

// Failed load condition testing
void forceImmediateCompletion();
@@ -2373,6 +2373,36 @@ bool TestController::hasDOMCache(WKStringRef origin)
return context.result;
}

struct FetchCacheSizeForOriginCallbackContext {
explicit FetchCacheSizeForOriginCallbackContext(TestController& controller)
: testController(controller)
{
}

TestController& testController;

bool done { false };
uint64_t result { 0 };
};

static void fetchCacheSizeForOriginCallback(uint64_t size, void* userData)
{
auto* context = static_cast<FetchCacheSizeForOriginCallbackContext*>(userData);
context->done = true;
context->result = size;
context->testController.notifyDone();
}

uint64_t TestController::domCacheSize(WKStringRef origin)
{
auto* dataStore = WKContextGetWebsiteDataStore(platformContext());
FetchCacheSizeForOriginCallbackContext context(*this);
WKWebsiteDataStoreGetFetchCacheSizeForOrigin(dataStore, origin, &context, fetchCacheSizeForOriginCallback);
if (!context.done)
runUntil(context.done, m_currentInvocation->shortTimeout());
return context.result;
}

#if !PLATFORM(COCOA) || !WK_API_ENABLED

void TestController::setStatisticsLastSeen(WKStringRef host, double seconds)
@@ -183,15 +183,16 @@ class TestController {
void statisticsClearThroughWebsiteDataRemoval();
void statisticsResetToConsistentState();

bool hasDOMCache(WKStringRef);

WKArrayRef openPanelFileURLs() const { return m_openPanelFileURLs.get(); }
void setOpenPanelFileURLs(WKArrayRef fileURLs) { m_openPanelFileURLs = fileURLs; }

void terminateNetworkProcess();

void removeAllSessionCredentials();

void clearDOMCache(WKStringRef origin);
bool hasDOMCache(WKStringRef origin);
uint64_t domCacheSize(WKStringRef origin);

private:
WKRetainPtr<WKPageConfigurationRef> generatePageConfiguration(WKContextConfigurationRef);
@@ -1219,6 +1219,15 @@ WKRetainPtr<WKTypeRef> TestInvocation::didReceiveSynchronousMessageFromInjectedB
return result;
}

if (WKStringIsEqualToUTF8CString(messageName, "DOMCacheSize")) {
ASSERT(WKGetTypeID(messageBody) == WKStringGetTypeID());
WKStringRef origin = static_cast<WKStringRef>(messageBody);

auto domCacheSize = TestController::singleton().domCacheSize(origin);
WKRetainPtr<WKTypeRef> result(AdoptWK, WKUInt64Create(domCacheSize));
return result;
}

ASSERT_NOT_REACHED();
return nullptr;
}

0 comments on commit 7ea502a

Please sign in to comment.