[LiveComponent] TemplateCacheWarmer not warming up cache

[althaus]

Hi,

I've got an issue warming up the cache on PROD:

/my/app # php bin/console c:c --no-warmup

 / Clearing the cache for the prod environment with debug false


 [OK] Cache for the "prod" environment (debug=false) was successfully cleared.


/my/app # php bin/console c:c

^ "/var/cache/my/app/live_components_twig_templates.map"

 / Clearing the cache for the prod environment with debug false

^ "/var/cache/my/app/serialization.php"
^ "/var/cache/my/app/validation.php"
^ "/var/cache/my/app/webpack_encore.cache.php"

 [OK] Cache for the "prod" environment (debug=false) was successfully cleared.

/my/app # php bin/console c:c

 / Clearing the cache for the prod environment with debug false

^ "/var/cache/my/app/serialization.php"
^ "/var/cache/my/app/validation.php"
^ "/var/cache/my/app/live_components_twig_templates.map"
^ "/var/cache/my/app/webpack_encore.cache.php"

 [OK] Cache for the "prod" environment (debug=false) was successfully cleared.

1. Clear the cache with no warmup
2. Run warmup
3. Run warmup again

The dump is the warmUp call of the PhpArrayAdapter and dumps $this->file.

In the first warmup the LiveComponent cache is warmed up before the cache get's cleared. I tracked this down to the container build process. Only on the 2nd run the warmer is called after clearing the cache. I've managed to track this to the isOptional=false setting in the LiveComponent's TemplateCacheWarmer and I'm now confused if this is a bug in Symfony, in the LiveComponent or somewhere in my app?

I'm running the latest Symfony 6.4 right now.

Kind regards
Matthias

[smnandre]

In your second example is everything working ? Or is the file destroyed ?

[althaus]

The /var/cache/my/app/live_components_twig_templates.map gets lost.

[smnandre]

Hmm.. @kbond maybe has a immediate idea ?

It seems very weird that the Clearing the cache for the prod environment with debug false is displayed after the cache warmer runs...

[smnandre]

Hmm i did not reproduce with the ux.symfony.com code

APP_ENV=prod APP_DEBUG=false sf c:c --no-warmup

APP_ENV=prod APP_DEBUG=false sf c:c 

APP_ENV=prod APP_DEBUG=false sf c:c 

Can you create a reproducer to isolate the problem and investigate ?

[althaus]

I've added some debugging and the stack trace tells me:

1. The Warmup is run by the container builder as it's marked as "not optional".
2. The ClearCacheCommand then removes the file and doesn't warm it up again.
3. The ClearCacheComannds finally warms it up.

Could it be that the file must be located in the build dir? We've got some strict directory rules as the app gets deployed through Debian packages and therefore cache and build are completley seperated.

[althaus]

Can you create a reproducer to isolate the problem and investigate ?

I'll try it later. My gut says that there's an issue if the cache dir is not the same as the build dir. Can you try to change those easily in your ux project? Otherwise I'll come back tomorrow on this.

[kbond]

Hmm, I didn't know this class existed. @sneakyvv, any ideas here?

[althaus]

Some more digging: I've reverted some changes in the app to get back to a working version. The last change I did not revert is the upgrade to Symfony 6. It seems that there's a compat issue.

public function warmUp(string $cacheDir, string $buildDir = null): array

Could it be related to the new $buildDir parameter? Should the warmer use this if it's given and not null?

[smnandre]

Yes it's probably related indeed... do the other cachewarmer use build dir now ? That what you say ?

[smnandre]

Well there is something not working there..

If i hardcode the build directory like this

// Kernel.php
class Kernel extends BaseKernel
{
    use MicroKernelTrait;

    public function getBuildDir(): string
    {
        // if (isset($_SERVER['APP_BUILD_DIR'])) {
        //     return $_SERVER['APP_BUILD_DIR'].'/'.$this->environment;
        // }

        return '/foo-bar/var/build/prod';
        //return $this->getProjectDir().'/var/build/'.$this->environment;
    }
}

I updated the Warmer to generate in both build and cache directories

And i added some "echo" lines to debug (in a dirty way, i know ... 😶‍🌫️ ).

First time i run APP_ENV=prod APP_DEBUG=true sf c:c

/foo-bar/var/build/prod
Cache: /foo-bar/var/cache/prod
CacheFile: /foo-bar/var/cache/prod/live/live_components_twig_templates.map
/foo-bar/var/cache/prod/live/live_components_twig_templates.map
1
Build: /foo-bar/var/build/prod
BuildFile: /foo-bar/var/build/prod/live/live_components_twig_templates.map
/foo-bar/var/build/prod/live/live_components_twig_templates.map
1

The second time.... APP_ENV=prod APP_DEBUG=true sf c:c

 // Clearing the cache for the prod environment with debug true                                                         

/foo-bar/var/build/pro_
Cache: /foo-bar/var/cache/prod
CacheFile: /foo-bar/var/cache/prod/live/live_components_twig_templates.map
not exists= /foo-bar/var/cache/prod/live
/foo-bar/var/cache/prod/live/live_components_twig_templates.map
1
Build: /foo-bar/var/build/pro_
BuildFile: /foo-bar/var/build/pro_/live/live_components_twig_templates.map
not exists= /foo-bar/var/build/pro_/live
/foo-bar/var/build/pro_/live/live_components_twig_templates.map
1
                                                                                                                        
 [OK] Cache for the "prod" environment (debug=true) was successfully cleared.     

I have no idea where this 'pro_' comes from

--

Moreover, i reproduced your problem locally, do we agree everything is OK if you start with a c:w instead of c:c ?

[smnandre]

Ok, the "pro_" is a legit step of the cache-clear.

When i hard-code some "$buildDir ?? $cacheDir" in the TemplateCacheWarmer it seems to work when build dir is different from cacheDir

But i'd like to understand what is happening before making changes (and we must be sure everything work for people without custom buildDir, etc etc..)

[althaus]

Regarding what Symfony does: I've only found the annotation.map in our build dir. Tracking this down leads to these lines:

https://github.com/symfony/symfony/blob/243db27c9b595a0a159dda577405fc36839465dd/src/Symfony/Bundle/FrameworkBundle/CacheWarmer/AnnotationsCacheWarmer.php#L50C91-L50C100

https://github.com/symfony/symfony/blob/243db27c9b595a0a159dda577405fc36839465dd/src/Symfony/Bundle/FrameworkBundle/CacheWarmer/AbstractPhpFileCacheWarmer.php#L40

So there a magical things happening when the additional - not yet required $buildDir parameter is given. It's somehow hard to track down the intent. In Symfony 7 the parameter is given. There I've found

https://github.com/symfony/symfony/blob/635a9f1bcc48386e5013bcf0454b5857d407cc78/src/Symfony/Bundle/FrameworkBundle/CacheWarmer/ConfigBuilderCacheWarmer.php#L49

Which is a non-optional cache warmer and this is using the build dir.

[smnandre]

When you have time, please tell me if #1525 works for you :)

[althaus]

When you have time, please tell me if #1525 works for you :)

I've just baked an update and the changes work with our setup... 🥳

acme ~ # ll /path/to/build/dir/cache/prod/
-rw-r--r-- 1 root root   5880 21. Feb 14:46 annotations.map
-rw-r--r-- 1 root root    851 21. Feb 14:46 App_KernelProdContainer.php
-rw-r--r-- 1 root root      0 21. Feb 14:46 App_KernelProdContainer.php.lock
-rw-r--r-- 1 root root 109076 21. Feb 14:46 App_KernelProdContainer.php.meta
-rw-r--r-- 1 root root  67313 21. Feb 14:46 App_KernelProdContainer.preload.php
drwxr-xr-x 2 root root  49152 21. Feb 14:49 ContainerYJNguxU
drwxr-xr-x 3 root root   4096 16. Feb 14:07 doctrine
-rw-r--r-- 1 root root  10336 21. Feb 14:46 live_components_twig_templates.map
drwxr-xr-x 3 root root   4096 16. Feb 14:07 Symfony

acme ~ # ll /var/cache/path/to/cachedir/
insgesamt 48
drwxrws---  4 app app  4096 21. Feb 14:53 pools
drwxrws---  2 app app  4096 21. Feb 14:50 templates
drwxr-sr-x 13 app app  4096 21. Feb 14:53 twig
-rw-r--r--  1 app app 11330 21. Feb 14:53 url_generating_routes.php
-rw-r--r--  1 app app  5632 21. Feb 14:53 url_generating_routes.php.meta
-rw-r--r--  1 app app  5953 21. Feb 14:53 url_matching_routes.php
-rw-r--r--  1 app app  5632 21. Feb 14:53 url_matching_routes.php.meta

The cache file is located in the build dir and happy survives any cache clearing.

[smnandre]

Perfect! Thank you for all the details, context, etc :) Makes it easier to debug!