Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add caching to service worker #92

Merged
merged 20 commits into from
Mar 14, 2024
Merged
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 76 additions & 2 deletions src/sw.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { createVerifiedFetch, type VerifiedFetch } from '@helia/verified-fetch'
import { HeliaServiceWorkerCommsChannel, type ChannelMessage } from './lib/channel.ts'
import { getConfig } from './lib/config-db.ts'
import { contentTypeParser } from './lib/content-type-parser.ts'
import { getSubdomainParts } from './lib/get-subdomain-parts.ts'
import { getSubdomainParts, type UrlParts } from './lib/get-subdomain-parts.ts'
import { isConfigPage } from './lib/is-config-page.ts'
import { error, log, trace } from './lib/logger.ts'
import { findOriginIsolationRedirect } from './lib/path-or-subdomain.ts'
Expand Down Expand Up @@ -40,6 +40,8 @@ interface GetVerifiedFetchUrlOptions {
declare let self: ServiceWorkerGlobalScope
let verifiedFetch: VerifiedFetch
const channel = new HeliaServiceWorkerCommsChannel('SW')
const IMMUTABLE_CACHE = 'IMMUTABLE_CACHE'
const MUTABLE_CACHE = 'MUTABLE_CACHE'
const urlInterceptRegex = [new RegExp(`${self.location.origin}/ip(n|f)s/`)]
const updateVerifiedFetch = async (): Promise<void> => {
verifiedFetch = await getVerifiedFetch()
Expand Down Expand Up @@ -85,6 +87,8 @@ self.addEventListener('fetch', (event) => {
const request = event.request
const urlString = request.url
const url = new URL(urlString)
const { protocol, id } = getSubdomainParts(event.request.url)
log('helia-sw: incoming request url: %s:', event.request.url, protocol, id)

if (isConfigPageRequest(url) || isSwAssetRequest(event)) {
// get the assets from the server
Expand All @@ -101,7 +105,46 @@ self.addEventListener('fetch', (event) => {
// intercept and do our own stuff...
event.respondWith(fetchHandler({ path: url.pathname, request }))
} else if (isSubdomainRequest(event)) {
event.respondWith(fetchHandler({ path: url.pathname, request }))
const isMutable = protocol === 'ipns'
const cacheKey = `${event.request.url}-${event.request.headers.get('Accept') ?? ''}`
const cacheName = isMutable ? MUTABLE_CACHE : IMMUTABLE_CACHE
log('helia-sw: cache name: %s | key: %s', cacheName, cacheKey)

event.respondWith((async () => {
2color marked this conversation as resolved.
Show resolved Hide resolved
const cache = await caches.open(cacheName)
const cachedResponse = await cache.match(cacheKey)

if ((cachedResponse != null) && !hasExpired(cachedResponse)) {
// If there is an entry in the cache for event.request,
// then response will be defined and we can just return it.
log('helia-ws: cached response HIT for %s (expires: %s) %o', cacheKey, cachedResponse.headers.get('Expires'), cachedResponse)

trace('helia-ws: updating cache for %s in the background', cacheKey)
// 👇 update cache in the background wihtout awaiting
fetchHandler({ path: url.pathname, request })
2color marked this conversation as resolved.
Show resolved Hide resolved

return cachedResponse
}

// 👇 fetch because no cached response was found
const response = await fetchHandler({ path: url.pathname, request })

if (response.ok) {
// 👇 only cache successful responses
log('helia-ws: storing cache key %s in cache', cacheKey)
// Clone the response since streams can only be consumed once.
const respToCache = response.clone()

if (isMutable) {
// 👇 Set expires header to an hour from now for mutable (ipns://) resources
setExpiryHeader(respToCache, 3600)
}

cache.put(cacheKey, respToCache)
}

return response
})())
}
})

Expand Down Expand Up @@ -177,6 +220,37 @@ function isSwAssetRequest (event: FetchEvent): boolean {
return isActualSwAsset
}

/**
* Set the expires header with a timestamp base
*/
function setExpiryHeader (response: Response, ttlSeconds: number = 3600): Response {
2color marked this conversation as resolved.
Show resolved Hide resolved
const expirationTime = new Date(Date.now() + ttlSeconds * 1000)

response.headers.set('Expires', expirationTime.toUTCString())
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be sensible to make this cache header explicitly SW specific (originally suggested in https://phyks.me/2019/01/manage-expiration-of-cached-assets-with-service-worker-caching.html)

Suggested change
response.headers.set('Expires', expirationTime.toUTCString())
response.headers.set('sw-cache-expires', expirationTime.toUTCString())

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that makes sense

Copy link
Member

@lidel lidel Mar 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long term, if we can avoid inventing things specific to service worker, and reuse HTTP semantics, that is better for verified-fetch being useful in more contexts, and easier to test (if these are set upstream, places like https://github.com/ipfs/helia-http-gateway get these headers for free, and we also increase surface of things that can be tested there).

My initial thought here was that if we reuse semantically meaningful Expires and set it only if not set yet, in the future it can be set inside verified-fetch, and not only we don't need to invent special header here, but also improve the upstream library and the code here will be smaller.

But fine to go with custom header here for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lidel Because the SW Cache API always takes precedence over browser HTTP cache and and verified-fetch will be returning the Cache-Control header, the Expires header would be: no-op.

Even if we were to use the Expires header, by respecting it when there's a Cache-Control header, we'd be breaking from HTTP semantics. This is why I think it makes sense to make it clear that this header is purely for SW purposes.

return response
}

/**
* Checks whether a cached response object has expired by looking at the expires header
* Note that this ignores the Cache-Control header since the expires header is set by us
*/
function hasExpired (response: Response): boolean {
const expiresHeader = response.headers.get('Expires')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const expiresHeader = response.headers.get('Expires')
const expiresHeader = response.headers.get('sw-cache-expires')

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved in #109


if (!expiresHeader) {
return false
}

const expires = new Date(expiresHeader)
const now = new Date()

if (expires < now) {
return true
}

return false
2color marked this conversation as resolved.
Show resolved Hide resolved
}

async function fetchHandler ({ path, request }: FetchHandlerArg): Promise<Response> {
/**
* > Any global variables you set will be lost if the service worker shuts down.
Expand Down
Loading