Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Precaching temp #1342

Merged
merged 5 commits into from Mar 5, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
42 changes: 34 additions & 8 deletions infra/testing/activate-sw.js
Expand Up @@ -6,17 +6,43 @@ module.exports = async (swUrl) => {
}

const error = await global.__workbox.webdriver.executeAsyncScript((swUrl, cb) => {
if (navigator.serviceWorker.controller &&
navigator.serviceWorker.controller.scriptURL === swUrl) {
cb();
} else {
navigator.serviceWorker.addEventListener('controllerchange', () => {
if (navigator.serviceWorker.controller.scriptURL === swUrl) {
cb();
function _onStateChangePromise(registration, desiredState) {
return new Promise((resolve, reject) => {
if (registration.installing === null) {
throw new Error('Service worker is not installing.');
}

let serviceWorker = registration.installing;

// We unregister all service workers after each test - this should
// always trigger an install state change
let stateChangeListener = function(evt) {
if (evt.target.state === desiredState) {
serviceWorker.removeEventListener('statechange', stateChangeListener);
resolve();
return;
}

if (evt.target.state === 'redundant') {
serviceWorker.removeEventListener('statechange', stateChangeListener);

// Must call reject rather than throw error here due to this
// being inside the scope of the callback function stateChangeListener
reject(new Error('Installing servier worker became redundant'));
return;
}
};

serviceWorker.addEventListener('statechange', stateChangeListener);
});
navigator.serviceWorker.register(swUrl).catch((error) => cb(error));
}

navigator.serviceWorker.register(swUrl)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The background on why we were waiting for the correct SW to take control, rather than waiting on activation, is in w3c/ServiceWorker#799 (comment)

We've seen flakiness in unit tests in the past due to the small window of time in between when a service worker activates and when it takes control of a window client. If the window client makes a network request in that window of time, the correct service worker won't intercept it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I saw flakiness the other way, so how about we check for both?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we dig into the flakiness you saw in the other direction? Potentially outside of this PR, if you can restore the previous logic and supplement it with the additional logic that's now necessarily.

FWIW, a scenario in which a SW can take control of a page before it's been activated sounds like a (potentially serious) browser bug.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spoke to @jakearchibald about this as I don't think it is a bug.

Having multiple evens, one claiming and others doing work means:

All clients will be claimed straight away, but the service worker won't activate until promises passed to waitUntil resolve

This makes sense to me but does bring in to question whether we should do some other magic around clientsClaim to account for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, really? My understanding was that the service worker couldn't take control of a client until after it activated. Huh.

.then((registration) => {
return _onStateChangePromise(registration, 'activated');
})
.then(() => cb())
.catch((err) => cb(err));
}, swUrl);

if (error) {
Expand Down
2 changes: 1 addition & 1 deletion packages/workbox-precaching/_default.mjs
Expand Up @@ -165,7 +165,7 @@ moduleExports.precache = (entries) => {
}));
});
self.addEventListener('activate', (event) => {
event.waitUntil(precacheController.cleanup());
event.waitUntil(precacheController.activate());
});
};

Expand Down
50 changes: 45 additions & 5 deletions packages/workbox-precaching/controllers/PrecacheController.mjs
Expand Up @@ -164,11 +164,15 @@ class PrecacheController {
}
}

// Clear any existing temp cache
await caches.delete(this._getTempCacheName());

const entriesToPrecache = [];
const entriesAlreadyPrecached = [];

for (const precacheEntry of this._entriesToCacheMap.values()) {
if (await this._precacheDetailsModel._isEntryCached(precacheEntry)) {
if (await this._precacheDetailsModel._isEntryCached(
this._cacheName, precacheEntry)) {
entriesAlreadyPrecached.push(precacheEntry);
} else {
entriesToPrecache.push(precacheEntry);
Expand All @@ -177,7 +181,7 @@ class PrecacheController {

// Wait for all requests to be cached.
await Promise.all(entriesToPrecache.map((precacheEntry) => {
return this._cacheEntry(precacheEntry, options.plugins);
return this._cacheEntryInTemp(precacheEntry, options.plugins);
}));

if (process.env.NODE_ENV !== 'production') {
Expand All @@ -190,6 +194,40 @@ class PrecacheController {
};
}

/**
* Takes the current set of temporary files and moves them to the final
* cache, deleting the temporary cache once copying is complete.
*
* @return {
* Promise<workbox.precaching.CleanupResult>}
* Resolves with an object containing details of the deleted cache requests
* and precache revision details.
*/
async activate() {
const tempCache = await caches.open(this._getTempCacheName());

const requests = await tempCache.keys();
await Promise.all(requests.map(async (request) => {
const response = await tempCache.match(request);
await cacheWrapper.put(this._cacheName, request, response);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have #1312 open to track the larger question of how to deal with quota issues that come up during precaching.

In the meantime, this PR introduces some overhead due to every response in the temp cache being duplicated first, and with all the temp entries only getting deleted after all of those duplicates are written.

What if you added in

await cacheWrapper.delete(this._getTempCacheName(), request);

inside this loop, to clear out the temporary copy of each individual response after it's been duplicated to the final cache?

You can still leave the

await caches.delete(this._getTempCacheName());

outside the loop, to delete the cache itself, though at that point the cache should have 0 entries in it.

}));

await caches.delete(this._getTempCacheName());

return this._cleanup();
}

/**
* Returns the name of the temporary cache.
*
* @return {string}
*
* @private
*/
_getTempCacheName() {
return `${this._cacheName}-temp`;
}

/**
* Requests the entry and saves it to the cache if the response
* is valid.
Expand All @@ -203,7 +241,7 @@ class PrecacheController {
* promise resolves with true if the entry was cached / updated and
* false if the entry is already cached and up-to-date.
*/
async _cacheEntry(precacheEntry, plugins) {
async _cacheEntryInTemp(precacheEntry, plugins) {
let response = await fetchWrapper.fetch(
precacheEntry._networkRequest,
null,
Expand All @@ -214,7 +252,7 @@ class PrecacheController {
response = await cleanRedirect(response);
}

await cacheWrapper.put(this._cacheName,
await cacheWrapper.put(this._getTempCacheName(),
precacheEntry._cacheRequest, response, plugins);

await this._precacheDetailsModel._addEntry(precacheEntry);
Expand All @@ -232,8 +270,10 @@ class PrecacheController {
* Promise<workbox.precaching.CleanupResult>}
* Resolves with an object containing details of the deleted cache requests
* and precache revision details.
*
* @private
*/
async cleanup() {
async _cleanup() {
const expectedCacheUrls = [];
this._entriesToCacheMap.forEach((entry) => {
const fullUrl = new URL(entry._cacheRequest.url, location).toString();
Expand Down
11 changes: 4 additions & 7 deletions packages/workbox-precaching/models/PrecachedDetailsModel.mjs
Expand Up @@ -15,7 +15,6 @@
*/

import {DBWrapper} from 'workbox-core/_private/DBWrapper.mjs';
import {cacheNames} from 'workbox-core/_private/cacheNames.mjs';
import '../_version.mjs';

// Allows minifier to mangle this name
Expand All @@ -32,12 +31,9 @@ class PrecachedDetailsModel {
/**
* Construct a new model for a specific cache.
*
* @param {string} cacheName
*
* @private
*/
constructor(cacheName) {
this._cacheName = cacheNames.getPrecacheName(cacheName);
constructor() {
this._db = new DBWrapper(`workbox-precaching`, 2, {
onupgradeneeded: this._handleUpgrade,
});
Expand Down Expand Up @@ -70,18 +66,19 @@ class PrecachedDetailsModel {
* Check if an entry is already cached. Returns false if
* the entry isn't cached or the revision has changed.
*
* @param {string} cacheName
* @param {PrecacheEntry} precacheEntry
* @return {boolean}
*
* @private
*/
async _isEntryCached(precacheEntry) {
async _isEntryCached(cacheName, precacheEntry) {
const revisionDetails = await this._getRevision(precacheEntry._entryId);
if (revisionDetails !== precacheEntry._revision) {
return false;
}

const openCache = await caches.open(this._cacheName);
const openCache = await caches.open(cacheName);
const cachedResponse = await openCache.match(precacheEntry._cacheRequest);
return !!cachedResponse;
}
Expand Down
5 changes: 4 additions & 1 deletion test/workbox-google-analytics/integration/basic-example.js
Expand Up @@ -27,11 +27,14 @@ describe(`[workbox-google-analytics] Load and use Google Analytics`,
data, [messageChannel.port2]);
};

beforeEach(async function() {
before(async function() {
// Load the page and wait for the first service worker to activate.
await driver.get(testingUrl);

await activateSW(swUrl);
});

beforeEach(async function() {
// Reset the spied requests array.
await driver.executeAsyncScript(messageSw, {
action: 'clear-spied-requests',
Expand Down
3 changes: 3 additions & 0 deletions test/workbox-precaching/integration/precache-and-update.js
Expand Up @@ -30,6 +30,8 @@ describe(`[workbox-precaching] Precache and Update`, function() {
const SW_1_URL = `${testingUrl}sw-1.js`;
const SW_2_URL = `${testingUrl}sw-2.js`;

await global.__workbox.webdriver.get(testingUrl);

const getIdbData = global.__workbox.seleniumBrowser.getId() === 'safari' ?
require('../utils/getPrecachedIDBData-safari') :
require('../utils/getPrecachedIDBData');
Expand Down Expand Up @@ -139,6 +141,7 @@ describe(`[workbox-precaching] Precache and Update`, function() {
// Refresh the page and test that the requests are as expected
global.__workbox.server.reset();
await global.__workbox.webdriver.get(testingUrl);

requestsMade = global.__workbox.server.getRequests();
// Ensure the HTML page is returned from cache and not network
expect(requestsMade['/test/workbox-precaching/static/precache-and-update/']).to.equal(undefined);
Expand Down