document workbox caching strategies

[tacman]

Description

Because AssetMapper generates a hash for the js/css files, they can be safely cached in the manifest. This is true for both dev and prod.

What's the configuration it for putting the /assets path in the manifest, so those assets are only loaded once?

[tacman]

I've read, and re-read, this: https://stackoverflow.com/questions/46830493/is-there-any-way-to-cache-all-files-of-defined-folder-path-in-service-worker

But am still trying to understand how phpwa.yaml works in conjunction with the manifest.

For example, in the phpwa-demo, there's a hello_controller.js in assets. Even though it's not being used in the demo, I'd like to cache it so if it is, it's available to be run offline.

[tacman]

Hmm, it looks like it does cache the controllers, so they're available offline.

How do I cache an API route? I want the app to grab the existing items (tasks) when first loading, and serve that or the most recent version when it's offline.

    serviceworker:
        enabled: true
        src: "sw.js"
        skip_waiting: true
        workbox:
            warm_cache_urls:
                - 'app_homepage'
                - '_api_/items{._format}_get_collection'
            page_fallback: 'app_homepage'

I added the route to warm_cache_urls, but that's not right. (Also, shouldn't it be called warm_cache_routes?)

[tacman]

https://developer.chrome.com/docs/workbox/modules/workbox-strategies

I guess what I'm looking for is how to map phpwa.yaml configuration to the workbox caching strategies.

[Spomky]

What's the configuration it for putting the /assets path in the manifest, so those assets are only loaded once?

The assets are not put into the manifest file. Those files are served by the service worker if Workbox is used.
The SW compiler will include all assets depending on the configurable regex. Note that it is only available for images and static assets (JS, JSON...).

For example, in the phpwa-demo, there's a hello_controller.js in assets.
As long as it is managed by AssetMapper, it is possible to precache it. This is the case for this file.

Hmm, it looks like it does cache the controllers, so they're available offline.
How do I cache an API route? I want the app to grab the existing items (tasks) when first loading, and serve that or the most recent version when it's offline.

There are two types of cached pages: the ones the user visits and the ones listed under this configuration key. You can use the route name (with or without parameters) or any paths.

I added the route to warm_cache_urls, but that's not right. (Also, shouldn't it be called warm_cache_routes?)
I do think it is the right way.
It can be routes or paths. I took this name from Workbox.

[tacman]

Thanks, that's helpful.

What do you think of allowing the user to define the caching strategy based on the url/route?

I think assets should be CacheOnly, and they should be cached on startup (because of the hash). That would get rid of a lot of extra network requests.

Then I'd like to be able to define the cache for the API calls. My route in the above example isn't caching at all ('api/items{._format}_get_collection'), is there something I need to do.

What do you think of a phpwa:dump or :debug to show some details? Where can I look to see what routes are cached and how?

[Spomky]

What do you think of allowing the user to define the caching strategy based on the url/route?
The user has full control on the service worker and is free to add custom routes or any other rules.

I think assets should be CacheOnly, and they should be cached on startup (because of the hash). That would get rid of a lot of extra network requests.
That was what I did in the early version of the bundle. This is true for all assets served by AssetMapper, but caused issues when fetching other assets such as JSON files (typically served by APIs. As they are CacheOnly, if the API state changes, the old data are served.

Then I'd like to be able to define the cache for the API calls. My route in the above example isn't caching at all ('api/items{._format}_get_collection'), is there something I need to do.
I will test on my side. The cached pages are visible in the debug window (F12 with most of the browsers).

What do you think of a phpwa:dump or :debug to show some details?
I like the idea. What type of data should I dump?

[tacman]

Is there a way to configure:

• /assets -> CacheOnly
• /api ->Network First, then Cache
• *.json CacheFirst, then Network

As far as dump, even dumping the service worker (with comments?) would be fine. I know its dynamically created, but I'm having a hard time even seeing it or debugging it, as I'm not sure what actually triggers the ServiceWorkerCompiler.

[Spomky]

Is there a way to configure:

It seems possible to me by adding several rules based on the path.

import {registerRoute} from 'workbox-routing';
import {CacheOnly, CacheFirst, NetworkFirst, StaleWhileRevalidate} from 'workbox-strategies';

registerRoute(({url}) => url.pathname.startsWith('/assets'), new CacheOnly());
registerRoute(({url}) => url.pathname.startsWith('/api'), new NetworkFirst());
registerRoute(({url}) => url.pathname.endsWith('.json'), new StaleWhileRevalidate());

As far as dump, even dumping the service worker (with comments?) would be fine.
Ok I will perform some experiments on this

I'm not sure what actually triggers the ServiceWorkerCompiler.
The bundle calls it when assetmap:compile is called. FIles are compiled with all other assests.

[tacman]

Since all assets have a hash, we don't need the stale-while-revalidate, it causes lots of network requests when the app is offline. I think /assets should be cache only, I think you already warm it up.

Static resources cache
The static resources cache recipe allows your service worker to respond to a request for static resources, specifically CSS, JavaScript, and Web Worker requests, with a stale-while-revalidate caching strategy so those assets can be quickly served from the cache and be updated in the background

[davidromani]

import {registerRoute} from 'workbox-routing';
import {CacheOnly, CacheFirst, NetworkFirst, StaleWhileRevalidate} from 'workbox-strategies';

registerRoute(({url}) => url.pathname.startsWith('/assets'), new CacheOnly());
registerRoute(({url}) => url.pathname.startsWith('/api'), new NetworkFirst());
registerRoute(({url}) => url.pathname.endsWith('.json'), new StaleWhileRevalidate());

I need to acheive a similar scenario that @tacman is exposing, but I'm quite confused about where to put this chunk of code. Can you specify where is the right place? Inside /assets/sw.js? Read a lot of information about Workbox and Service Workers in general and I'm not able to understand how to merge it.

[tacman]

What about an attribute with an optional caching strategy?

#[PWA(cachingStrategy: PWA::CacheFirst)]
#[Route('/terms')]

OR

#[Route('/terms', ['options' => ['pwa_strategy' => PWA::CacheOnly)]

If an attribute, it can even be done at the class level, then all controllers in that class would have the same strategy.

#[PWA(cachingStrategy: PWA::CacheFirst)]
class PageController extends AbstractController 

I wrote something similar (checks attributes at the class and route level), and could submit a similar PR if you're interested.

https://github.com/survos/BootstrapBundle/blob/main/src/SurvosBootstrapBundle.php#L66-L106

You'd have to do the magic of injecting the routes with the approriate cache strategy, as I haven't figured that out yet! Obviously, it's an enhancement to listing the routes/urls or adding the preg, both of which seem to be somewhat complicated at the moment, though could be fixed with documentation.

[Spomky]

Since all assets have a hash, we don't need the stale-while-revalidate, it causes lots of network requests when the app is offline. I think /assets should be cache only, I think you already warm it up.

In the last bug release 1.0.4, I reviewed the way assets/images/fonts are cached. stale-while-revalidate strategy is gone. This should solve the mp3 file issue (TBC).

How do I cache an API route?

Your configuration file is correct and I can see the route in the cache list.
However, the page cache is supposed to cache pages the user navigates to. Not the API calls.
I will add a dedicated rule so that /api can be cached (next minor release 1.1.0).

What about an attribute with an optional caching strategy?

I really like it. Having an attribute at the same level of a route looks great. I am not sure how to manage that for now...

[tacman]

@davidromani I didn't realize either that you can simply add your own javascript to the sw.js file.

I'll add an issue to request that pwa:create:sw generate a more complete file with some comments.

[Spomky]

I'll add an issue to request that pwa:create:sw generate a more complete file with some comments.

This is a good idea. The command just copy-paste the file src/Resources/sw-skeleton.js

[Spomky]

Hi @tacman,

Can you explain how attribute detection works?

• How to now the attribute PWA and Route are set together on the class/method?
• The routes may have parameters. In particular, I often use /{_locale} at class level. How the attribute can guess what _locale means? Do we need to pass all possible values to the PWA attributes?
• The caching system uses regex in general. How to convert to a regex?

With all those questions in mind, I am wondering if the way to go be simplified. For example and to be in line with the existing caching system from the configuration file:

#[PWA(strategy: 'staleWhileRevalidate', regex: '\/foo/bar/\d+$', cacheName: 'foobars', broadcast: true, broadcastChannel: 'foobar-updates')]
class PageController extends AbstractController 

In this case, we only need to find all PWA attributes and convert them into a PageCache object.
Is it possible?

[tacman]

Since I'm particularly interested in offline capacity, I was mostly thinking of the attributes to facilitate warming the URL cache. To that end:

• Any route appropriately tagged that has no parameters can be added to the warmCacheUrls.
• An event subscriber that is customized by the user to generate urls (or route/path and params) and pass them to the Compiler.

The regular caching strategies (not focused on warming the cache), my idea for the attribute would be to find the route and parameters and pass them to the compiler, so that

#[Pwa(strategy: 'NetworkOnly')
#[Route('/onlineStatus', name: 'app_status')

#[Pwa(strategy: 'NetworkThenCache')
#[Route('/{_locale}/projectStatus/{projectCode}', name: 'project_status')

would be the same as

pwa:
    serviceworker:
        enabled: true
        src: "sw.js"
        workbox:
            page_caches:
              NetworkThenCache:
                urls:
                     - 'project_status' # Any route that matches, regardless of parameters???

              NetworkOnly:
                enabled: true
                network_timeout: 30
                urls:
                    - 'app_status' # Simple route name

I'm pretty sure we can pass the route match regex from the Symfony router to the compiler as well.

[tacman]

I'll mock something up for the attributes and put it in the debug toolbar, all the attributes are processed during the compiler pass and saved to /var/cache.

[tacman]

How to now the attribute PWA and Route are set together on the class/method?

https://github.com/survos/SurvosPwaExtraBundle/blob/main/src/SurvosPwaExtraBundle.php#L89-L129

[tacman]

Are you familiar with using the CacheWarmer? symfony/symfony#54237

I started the implementation (evidently the compilerPass in the wrong place for what I'm doing), I suspect I'll eventually figure it out.

[Spomky]

I close this issue as the page cahce evolved a lot. I opened a dedicated one for the CacheWarmer question.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.