feature #50112 [Asset] [AssetMapper] New AssetMapper component: Map a…

…ssets to publicly available, versioned paths (weaverryan)

This PR was squashed before being merged into the 6.3 branch.

Discussion
----------

[Asset] [AssetMapper] New AssetMapper component: Map assets to publicly available, versioned paths

| Q             | A
| ------------- | ---
| Branch?       | 6.3
| Bug fix?      | no
| New feature?  | yes
| Deprecations? | no
| Tickets       | Partner of #48371
| License       | MIT
| Doc PR        | TODO

Hi!

This will partners with and includes the importmaps PR #48371 (so that will no longer be needed). The goal is to allow you to write modern JavaScript & CSS, without needing a build system. This idea comes from Rails: https://github.com/rails/propshaft - and that heavily inspires this PR.

Example app using this: https://github.com/weaverryan/testing-asset-pipeline

Here's how it works:

A) You activate the asset mapper:

```yml
framework:
    asset_mapper:
        paths: ['assets/']
```

B) You put some files into your `assets/` directory (which sits at the root of your project - exactly like now with Encore). For example, you might create an `assets/app.js`, `assets/styles/app.css` and `assets/images/skeletor.jpg`.

C) Refer to those assets with the normal `asset()` function

```twig
<link rel="stylesheet" href="{{ asset('styles/app.css') }}">
<script src="{{ asset('app.js') }}" defer></script>

<img src="{{ asset('images/skeletor.jpg') }}">
```

That's it! The final paths will look like this:

```html
<link rel="stylesheet" href="/assets/styles/app-b93e5de06d9459ec9c39f10d8f9ce5b2.css">
<script src="/assets/app-1fcc5be55ce4e002a3016a5f6e1d0174.js" defer type="module"></script>
<img src="/assets/images/skeletor-3f24cba25ce4e114a3116b5f6f1d2159.jpg">
```

How does that work?

* In the `dev` environment, a controller (technically a listener) intercepts the requests starting with `/assets/`, finds the file in the source `/assets/` directory, and returns it.
* In the `prod` environment, you run a `assets:mapper:compile` command, which copies all of the assets into `public/assets/` so that the real files are returned. It also dumps a `public/assets/manifest.json` so that the source paths (eg. `styles/app.css`) can be exchanged for their final paths super quickly.

### Extras Asset Compilers

There is also an "asset" compiler system to do some minor transformations in the source files. There are 3 built-in compilers:

A) `CssAssetUrlCompiler` - finds `url()` inside of CSS files and replaces with the real, final path - e.g. `url(../images/skeletor.jpg')` becomes `url(/assets/images/skeletor-3f24cba25ce4e114a3116b5f6f1d2159.jpg)` - logic taken from Rails
B) `SourceMappingUrlsCompiler` - also taken from Rails - if the CSS file already contains a sourcemap URL, this updates it in the same way as above (Note: actually ADDING sourcemaps is not currently supported)
C) `JavaScriptImportPathCompiler` - experimental (I wrote the logic): replaces relative imports in JavaScript files `import('./other.js')` with their final path - e.g. `import('/assets/other.123456abcdef.js')`.

### Importmaps

This PR also includes an "importmaps" functionality. You can read more about that in #48371. In short, in your code you can code normally - importing "vendor" modules and your own modules:

```
// assets/app.js

import { Application } from '`@hotwired`/stimulus';
import CoolStuff from './cool_stuff.js';
```

Out-of-the-box, your browser won't know where to load ``@hotwired`/stimulus` from. To help it, run:

```
./bin/console importmap:require '`@hotwired`/stimulus';
```

This will updated/add an `importmap.php` file at the root of your project:

```php
return [
    'app' => [
        'path' => 'app.js',
        'preload' => true,
    ],
    '`@hotwired`/stimulus' => [
        'url' => 'https://ga.jspm.io/npm:`@hotwired`/stimulus@3.2.1/dist/stimulus.js',
    ],
];
```

In your `base.html.twig`, you add: `{{ importmap() }}` inside your `head` tag. The result is something like this:

```

<script type="importmap">{"imports": {
    "app": "/assets/app-cf9cfe84e945a554b2f1f64342d542bc.js",
    "cool_stuff.js": "/assets/cool_stuff-10b27bd6986c75a1e69c8658294bf22c.js",
    "`@hotwired`/stimulus": "https://ga.jspm.io/npm:`@hotwired`/stimulus@3.2.1/dist/stimulus.js",
}}</script>
</script>
<link rel="modulepreload" href="/assets/app-cf9cfe84e945a554b2f1f64342d542bc.js">
<link rel="modulepreload" href="/assets/cool_stuff-10b27bd6986c75a1e69c8658294bf22c.js">
<script type="module">import 'app';</script>
```

A few important things:

~~A) In the final `assets/app.js`, the `import CoolStuff from './cool_stuff';` will change to `import CoolStuff from './cool_stuff.js';` (the `.js` is added)~~
B) When your browser parses the final `app.js` (i.e. `/assets/app-cf9cfe84e945a554b2f1f64342d542bc.js`), when it sees the import for `./cool_stuff.js` it will then use the `importmap` above to find the real path and download it. It does the same thing when it sees the import for ``@hotwired`/stimulus`.
C) Because `app.js` has `preload: true` inside `importmap.php`, it (and anything it or its dependencies import) will also be preloaded - i.e. the `link rel="modulepreload"` will happen. This will tell your browser to download `app.js` and `cool_stuff.js` immediately. The is helpful for `cool_stuff.js` because we don't want to wait for the browser to download `app.js` and THEN realize it needs to download `cool_stuff.js`.

There is also an option to `--download` CDN paths to your local machine.

### Path "Namespaces" and Bundle Assets

You can also give each "path: in the mapper a "namespace" - e.g. an alternative syntax to the config is:

```yml
framework:
    asset_mapper:
        paths:
            assets: ''
            other_assets: 'other_namespace'
```

In this case, if there is an `other_assets/foo.css` file, then you can use `{{ asset('other_namespace/foo.css') }}` to get a path to it.

In practice, users won't do this. However, this DOES automatically add the `Resources/public/` or `public/` directory of every bundle as a "namespaced" path. For example, in EasyAdminBundle, the following code could be used:

```
<script src="{{ asset('bundles/easyadmin/login.js') }}"></script>
```

(Note: EA has some fancy versioning, but on a high-level, this is all correct). This would already work today thanks to `assets:install`. But as soon as this code is used in an app where the mapper is activated, the mapper would take over and would output a versioned filename - e.g.

```
<script src="/assets/bundles/easyadmin/login12345abcde.js"></script>
```

**OPEN QUESTIONS / NOTES**

* Only the "default" asset package uses the mapper. Extend to all?

TODO:

* [x] Twig importmap() extension needs a test
* [x] Need a way to allow a bundle to hook into `importmap()` - e.g. to add `data-turbo-track` on the `script` tags.
* [x] Make the AssetMapper have lazier dependencies

There are also a number of smaller things that we probably need at some point:

A)  a way to "exclude" paths from the asset map
~~B) a cache warmer (or something) to generate the `importmap` on production~~
C) perhaps a smart caching and invalidation system for the contents of assets - this would be for dev only - e.g. on every page load, we shouldn't need to calculate the contents of EVERY file in order to get its public path. If only cool_stuff.js was updated, we should only need to update its contents to get its path.
D) `debug:pipeline` command to show paths
E) Perhaps an `{{ asset_preload('styles/app.css') }}` Twig tag to add `<link rel="modulepreload">` for non-JS assets. This would also add `modulepreload` links for any CSS dependencies (e.g. if `styles/app.css` ``@import``s another CSS file, that would also be preloaded).

Cheers!

Commits
-------

e71a3a1 [Asset] [AssetMapper] New AssetMapper component: Map assets to publicly available, versioned paths

composer.json

@@ -58,6 +58,7 @@
     },
     "replace": {
         "symfony/asset": "self.version",
+        "symfony/asset-mapper": "self.version",
         "symfony/browser-kit": "self.version",
         "symfony/cache": "self.version",
         "symfony/clock": "self.version",

src/Symfony/Bridge/Twig/Extension/ImportMapExtension.php

@@ -0,0 +1,28 @@
+<?php
+
+/*
+ * This file is part of the Symfony package.
+ *
+ * (c) Fabien Potencier <fabien@symfony.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Symfony\Bridge\Twig\Extension;
+
+use Twig\Extension\AbstractExtension;
+use Twig\TwigFunction;
+
+/**
+ * @author Kévin Dunglas <kevin@dunglas.dev>
+ */
+final class ImportMapExtension extends AbstractExtension
+{
+    public function getFunctions(): array
+    {
+        return [
+            new TwigFunction('importmap', [ImportMapRuntime::class, 'importmap'], ['is_safe' => ['html']]),
+        ];
+    }
+}

src/Symfony/Bridge/Twig/Extension/ImportMapRuntime.php

@@ -0,0 +1,29 @@
+<?php
+
+/*
+ * This file is part of the Symfony package.
+ *
+ * (c) Fabien Potencier <fabien@symfony.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Symfony\Bridge\Twig\Extension;
+
+use Symfony\Component\AssetMapper\ImportMap\ImportMapRenderer;
+
+/**
+ * @author Kévin Dunglas <kevin@dunglas.dev>
+ */
+class ImportMapRuntime
+{
+    public function __construct(private readonly ImportMapRenderer $importMapRenderer)
+    {
+    }
+
+    public function importmap(?string $entryPoint = 'app'): string
+    {
+        return $this->importMapRenderer->render($entryPoint);
+    }
+}

src/Symfony/Bridge/Twig/Tests/Extension/ImportMapExtensionTest.php

@@ -0,0 +1,49 @@
+<?php
+
+/*
+ * This file is part of the Symfony package.
+ *
+ * (c) Fabien Potencier <fabien@symfony.com>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Extension;
+
+use PHPUnit\Framework\TestCase;
+use Symfony\Bridge\Twig\Extension\ImportMapExtension;
+use Symfony\Bridge\Twig\Extension\ImportMapRuntime;
+use Symfony\Component\AssetMapper\ImportMap\ImportMapRenderer;
+use Twig\Environment;
+use Twig\Loader\ArrayLoader;
+use Twig\RuntimeLoader\RuntimeLoaderInterface;
+
+class ImportMapExtensionTest extends TestCase
+{
+    public function testItRendersTheImportmap()
+    {
+        $twig = new Environment(new ArrayLoader([
+            'template' => '{{ importmap("application") }}',
+        ]), ['debug' => true, 'cache' => false, 'autoescape' => 'html', 'optimizations' => 0]);
+        $twig->addExtension(new ImportMapExtension());
+        $importMapRenderer = $this->createMock(ImportMapRenderer::class);
+        $expected = '<script type="importmap">{ "imports": {}}</script>';
+        $importMapRenderer->expects($this->once())
+            ->method('render')
+            ->with('application')
+            ->willReturn($expected);
+        $runtime = new ImportMapRuntime($importMapRenderer);
+
+        $mockRuntimeLoader = $this->createMock(RuntimeLoaderInterface::class);
+        $mockRuntimeLoader
+            ->method('load')
+            ->willReturnMap([
+                [ImportMapRuntime::class, $runtime],
+            ])
+        ;
+        $twig->addRuntimeLoader($mockRuntimeLoader);
+
+        $this->assertSame($expected, $twig->render('template'));
+    }
+}

src/Symfony/Bridge/Twig/composer.json

@@ -26,6 +26,7 @@
         "league/html-to-markdown": "^5.0",
         "phpdocumentor/reflection-docblock": "^3.0|^4.0|^5.0",
         "symfony/asset": "^5.4|^6.0",
+        "symfony/asset-mapper": "^6.3",
         "symfony/dependency-injection": "^5.4|^6.0",
         "symfony/finder": "^5.4|^6.0",
         "symfony/form": "^6.3",

src/Symfony/Bundle/FrameworkBundle/Controller/AbstractController.php

@@ -14,6 +14,7 @@
 use Psr\Container\ContainerInterface;
 use Psr\Link\EvolvableLinkInterface;
 use Psr\Link\LinkInterface;
+use Symfony\Component\AssetMapper\ImportMap\ImportMapManager;
 use Symfony\Component\DependencyInjection\Exception\ServiceNotFoundException;
 use Symfony\Component\DependencyInjection\ParameterBag\ContainerBagInterface;
 use Symfony\Component\Form\Extension\Core\Type\FormType;
@@ -44,6 +45,7 @@
 use Symfony\Component\WebLink\EventListener\AddLinkHeaderListener;
 use Symfony\Component\WebLink\GenericLinkProvider;
 use Symfony\Component\WebLink\HttpHeaderSerializer;
+use Symfony\Component\WebLink\Link;
 use Symfony\Contracts\Service\Attribute\Required;
 use Symfony\Contracts\Service\ServiceSubscriberInterface;
 use Twig\Environment;
@@ -95,6 +97,7 @@ public static function getSubscribedServices(): array
             'security.csrf.token_manager' => '?'.CsrfTokenManagerInterface::class,
             'parameter_bag' => '?'.ContainerBagInterface::class,
             'web_link.http_header_serializer' => '?'.HttpHeaderSerializer::class,
+            'asset_mapper.importmap.manager' => '?'.ImportMapManager::class,
         ];
     }
 
@@ -409,7 +412,7 @@ protected function addLink(Request $request, LinkInterface $link): void
     /**
      * @param LinkInterface[] $links
      */
-    protected function sendEarlyHints(iterable $links, Response $response = null): Response
+    protected function sendEarlyHints(iterable $links = [], Response $response = null, bool $preloadJavaScriptModules = false): Response
     {
         if (!$this->container->has('web_link.http_header_serializer')) {
             throw new \LogicException('You cannot use the "sendEarlyHints" method if the WebLink component is not available. Try running "composer require symfony/web-link".');
@@ -418,6 +421,17 @@ protected function sendEarlyHints(iterable $links, Response $response = null): R
         $response ??= new Response();
 
         $populatedLinks = [];
+
+        if ($preloadJavaScriptModules) {
+            if (!$this->container->has('asset_mapper.importmap.manager')) {
+                throw new \LogicException('You cannot use the JavaScript modules method if the AssetMapper component is not available. Try running "composer require symfony/asset-mapper".');
+            }
+
+            foreach ($this->container->get('asset_mapper.importmap.manager')->getModulesToPreload() as $url) {
+                $populatedLinks[] = new Link('modulepreload', $url);
+            }
+        }
+
         foreach ($links as $link) {
             if ($link instanceof EvolvableLinkInterface && !$link->getRels()) {
                 $link = $link->withRel('preload');

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Compiler/UnusedTagsPass.php

@@ -24,6 +24,7 @@ class UnusedTagsPass implements CompilerPassInterface
     private const KNOWN_TAGS = [
         'annotations.cached_reader',
         'assets.package',
+        'asset_mapper.compiler',
         'auto_alias',
         'cache.pool',
         'cache.pool.clearer',

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/Configuration.php

@@ -16,6 +16,8 @@
 use Psr\Log\LogLevel;
 use Symfony\Bundle\FullStack;
 use Symfony\Component\Asset\Package;
+use Symfony\Component\AssetMapper\AssetMapper;
+use Symfony\Component\AssetMapper\ImportMap\ImportMapManager;
 use Symfony\Component\Cache\Adapter\DoctrineAdapter;
 use Symfony\Component\Config\Definition\Builder\ArrayNodeDefinition;
 use Symfony\Component\Config\Definition\Builder\NodeBuilder;
@@ -161,6 +163,7 @@ public function getConfigTreeBuilder(): TreeBuilder
         $this->addSessionSection($rootNode);
         $this->addRequestSection($rootNode);
         $this->addAssetsSection($rootNode, $enableIfStandalone);
+        $this->addAssetMapperSection($rootNode, $enableIfStandalone);
         $this->addTranslatorSection($rootNode, $enableIfStandalone);
         $this->addValidationSection($rootNode, $enableIfStandalone);
         $this->addAnnotationsSection($rootNode, $willBeAvailable);
@@ -810,6 +813,97 @@ private function addAssetsSection(ArrayNodeDefinition $rootNode, callable $enabl
         ;
     }
 
+    private function addAssetMapperSection(ArrayNodeDefinition $rootNode, callable $enableIfStandalone): void
+    {
+        $rootNode
+            ->children()
+                ->arrayNode('asset_mapper')
+                    ->info('Asset Mapper configuration')
+                    ->{$enableIfStandalone('symfony/asset-mapper', AssetMapper::class)}()
+                    ->fixXmlConfig('path')
+                    ->fixXmlConfig('extension')
+                    ->fixXmlConfig('importmap_script_attribute')
+                    ->children()
+                        // add array node called "paths" that will be an array of strings
+                        ->arrayNode('paths')
+                            ->info('Directories that hold assets that should be in the mapper. Can be a simple array of an array of ["path/to/assets": "namespace"]')
+                            ->example(['assets/'])
+                            ->normalizeKeys(false)
+                            ->useAttributeAsKey('namespace')
+                            ->beforeNormalization()
+                                ->always()
+                                ->then(function ($v) {
+                                    $result = [];
+                                    foreach ($v as $key => $item) {
+                                        // "dir" => "namespace"
+                                        if (\is_string($key)) {
+                                            $result[$key] = $item;
+
+                                            continue;
+                                        }
+
+                                        if (\is_array($item)) {
+                                            // $item = ["namespace" => "the/namespace", "value" => "the/dir"]
+                                            $result[$item['value']] = $item['namespace'] ?? '';
+                                        } else {
+                                            // $item = "the/dir"
+                                            $result[$item] = '';
+                                        }
+                                    }
+
+                                    return $result;
+                                })
+                            ->end()
+                            ->prototype('scalar')->end()
+                        ->end()
+                        ->booleanNode('server')
+                            ->info('If true, a "dev server" will return the assets from the public directory (true in "debug" mode only by default)')
+                            ->defaultValue($this->debug)
+                        ->end()
+                        ->scalarNode('public_prefix')
+                            ->info('The public path where the assets will be written to (and served from when "server" is true)')
+                            ->defaultValue('/assets/')
+                        ->end()
+                        ->booleanNode('strict_mode')
+                            ->info('If true, an exception will be thrown if an asset cannot be found when imported from JavaScript or CSS files - e.g. "import \'./non-existent.js\'"')
+                            ->defaultValue(true)
+                        ->end()
+                        ->arrayNode('extensions')
+                            ->info('Key-value pair of file extensions set to their mime type.')
+                            ->normalizeKeys(false)
+                            ->useAttributeAsKey('extension')
+                            ->example(['.zip' => 'application/zip'])
+                            ->prototype('scalar')->end()
+                        ->end()
+                        ->scalarNode('importmap_path')
+                            ->info('The path of the importmap.php file.')
+                            ->defaultValue('%kernel.project_dir%/importmap.php')
+                        ->end()
+                        ->scalarNode('importmap_polyfill')
+                            ->info('URL of the ES Module Polyfill to use, false to disable. Defaults to using a CDN URL.')
+                            ->defaultValue(null)
+                        ->end()
+                        ->arrayNode('importmap_script_attributes')
+                            ->info('Key-value pair of attributes to add to script tags output for the importmap.')
+                            ->normalizeKeys(false)
+                            ->useAttributeAsKey('key')
+                            ->example(['data-turbo-track' => 'reload'])
+                            ->prototype('scalar')->end()
+                        ->end()
+                        ->scalarNode('vendor_dir')
+                            ->info('The directory to store JavaScript vendors.')
+                            ->defaultValue('%kernel.project_dir%/assets/vendor')
+                        ->end()
+                        ->scalarNode('provider')
+                            ->info('The provider (CDN) to use', class_exists(ImportMapManager::class) ? sprintf(' (e.g.: "%s").', implode('", "', ImportMapManager::PROVIDERS)) : '.')
+                            ->defaultValue('jspm')
+                        ->end()
+                    ->end()
+                ->end()
+            ->end()
+        ;
+    }
+
     private function addTranslatorSection(ArrayNodeDefinition $rootNode, callable $enableIfStandalone): void
     {
         $rootNode