[Asset] Add support for preloading with links and HTTP/2 push #21478

dunglas · 2017-01-31T16:02:23Z

Q	A
Branch?	master
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	n/a
License	MIT
Doc PR	symfony/symfony-docs#10309

Allows compatible clients to preload mandatory assets like scripts, stylesheets or images according to the "preload" working draft of the W3C.
Thanks to this PR, Symfony will automatically adds Link HTTP headers with a preload relation for mandatory assets. If an intermediate proxy supports HTTP/2 push, it will convert preload headers. For instance Cloudflare supports this feature.

It dramatically increases pages speed and make the web greener because only one TCP connection is used to fetch all mandatory assets (decrease servers and devices loads, improve battery lives).

Usage:

Updated version:

<html>
    <body>
    Hello
    <script src="{{ preload(asset('/scripts/foo.js'), 'script') }}"></script>
    </body>
</html>

~~First proposal:~~

<html>
    <body>
    Hello
    <script src="{{ preloaded_asset('/scripts/foo.js', 'script') }}"></script>
    </body>
</html>

Add tests

dunglas · 2017-01-31T21:01:15Z

Test and nopush support added.

Status: needs review

pkruithof · 2017-02-01T06:27:03Z

Very nice! We were about to implement this ourselves.

Why use a different function for this? Would an attribute not be more logical?

{{ asset('/scripts/foo.js', 'script', {preload: true}) }}

lyrixx · 2017-02-01T08:28:09Z

src/Symfony/Component/Asset/Preload/HttpFoundationPreloadManager.php

+ */
+class HttpFoundationPreloadManager implements PreloadManagerInterface
+{
+    private $resources = array();


Can you get rides of this state? Because right now it's a memory leak IIUC.

No it's not possible because this state gather all assets to add to the link.
However I can add a new method to clear it that will be called in the listener to avoid the memory leak.

we indeed need to clear the state. Otherwise, using the same kernel to handle multiple requests would leak resources from the previous request. Your implementation breaks the isolation of request handling

javiereguiluz · 2017-02-01T08:49:49Z

I have the same question as @pkruithof: why adding a new preloaded_asset() function instead of adding config options (globally for asset package config and locally for the asset() function).

dunglas · 2017-02-01T13:04:52Z

@pkruithof @javiereguiluz it was my 1st though, however the current signature is: { asset('/path', 'packageName') }.
Both { asset('/path', null, true, 'script', false) } and { asset('/path', null, {'preload': true, 'nopush': true, 'as': 'script') } look bad.
Changing parameters order isn't possible for BC. It's why I've introduced this new tag.

javiereguiluz · 2017-02-01T14:02:06Z

@dunglas thanks for the explanation. Another question: should we name this function asset_preload() instead?

First, it would match the naming followed by other functions, where the first word is "the common thing" (e.g. render_*(), form_*(), asset_*(), etc.)

Second, it would look better when using composition:

{{ preloaded_asset(asset('/scripts/foo.js'), 'script') }}
{{ asset_preload(asset('/scripts/foo.js'), 'script') }}

stof · 2017-02-01T14:06:40Z

src/Symfony/Component/Asset/EventListener/PreloadListener.php

+ *
+ * @author Kévin Dunglas <dunglas@gmail.com>
+ */
+class PreloadListener


should be an event subscriber IMO, like our other listeners, to embed the knownledge about the event being listened to

stof · 2017-02-01T14:07:49Z

src/Symfony/Component/Asset/Package.php

+        }
+
+        $url = $this->getUrl($path);
+        $this->preloadManager->addResource($url, $as, $nopush);


should the package really be the part being aware of the PreloadManager ? I don't think so. IMO, it should be done at a higher level in the stack. The preloading part is totally independent from the asset url building anyway

stof · 2017-02-01T14:10:13Z

src/Symfony/Component/Asset/Preload/HttpFoundationPreloadManager.php

+     *
+     * @param Response $response
+     */
+    public function setLinkHeader(Response $response)


a more reusable API would be to have a method returning the header instead, without dependency on the Response. The event listener would then handle the binding to HttpFoundation by setting the header (allowing the same manager to be reused by people using a PSR-7 stack instead, without reimplementing all the logic in a different class just because of this method).
Currently, the most important part of the business logic is in this method, which is not part of the interface

stof · 2017-02-01T14:12:54Z

src/Symfony/Component/Asset/Preload/HttpFoundationPreloadManager.php

+ */
+class HttpFoundationPreloadManager implements PreloadManagerInterface
+{
+    private $resources = array();


we indeed need to clear the state. Otherwise, using the same kernel to handle multiple requests would leak resources from the previous request. Your implementation breaks the isolation of request handling

dunglas · 2017-02-01T17:55:52Z

@stof (thanks!!), @lyrixx and @javiereguiluz comments took into account:

The preload feature is now 100% independent of the packages and has a Twig function of its own.
Dealing with the HttpFoundation's Response is only done in the listener (the manager can now be reused with any HTTP message implementation)
Memory leak fixed

The new syntax:

<html>
    <body>
    Hello
    <script src="{{ preload(asset('/scripts/foo.js'), 'script') }}"></script>
    </body>
</html>

stof · 2017-02-01T17:59:58Z

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

@@ -765,6 +767,15 @@ private function registerAssetsConfiguration(array $config, ContainerBuilder $co
            $defaultVersion = $this->createVersion($container, $config['version'], $config['version_format'], '_default');
        }

+        if (class_exists(PreloadManager::class)) {


missing class existence resource (or you could just conflict with asset < 3.3, and have all this in the XML file directly)

I don't get what you mean. But if it's ok to bump the dependency, I'll move this definition to the XML, it's cleaner.

stof · 2017-02-01T18:01:20Z

src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php


-        return $package
+        $package


Why editing this code ?

I've done that in a previous commit and it's no useless but I find the new version cleaner/simpler. I can revert it if you prefer the old one.

changing this without a need for it makes it harder to merge branches together for no reason

stof · 2017-02-01T18:01:46Z

src/Symfony/Bundle/FrameworkBundle/Tests/DependencyInjection/FrameworkExtensionTest.php

@@ -375,6 +376,16 @@ public function testAssetsDefaultVersionStrategyAsService()
        $this->assertEquals('assets.custom_version_strategy', (string) $defaultPackage->getArgument(1));
    }

+    public function testAssetHasPreloadListener()
+    {
+        if (!class_exists(PreloadListener::class)) {


composer ensures that this is always existing

Not if a version for symfony/asset lesser than 3.3 has been installed, right?

it cannot, due to the require-dev constraint

stof · 2017-02-01T18:03:18Z

src/Symfony/Component/Asset/EventListener/PreloadListener.php

+
+    public function onKernelResponse(FilterResponseEvent $event)
+    {
+        if ($value = $this->preloadManager->getLinkValue()) {


this logic must be done only for the master request.
An asset referenced in a subrequest must be preloaded by the master request, as it is the one being sent to the user, and your code would also use all previous master preloaded asset for the subrequest, thus breaking preloading.

stof · 2017-02-01T18:04:42Z

src/Symfony/Component/Asset/Preload/PreloadManager.php

+        foreach ($this->resources as $uri => $options) {
+            $part = "<$uri>; rel=preload";
+            if ('' !== $options['as']) {
+                $part .= "; as={$options['as']}";


We generally use sprintf rather than interpolation (even more when it is not simple interpolation)

I've done it this way because of https://blog.blackfire.io/php-7-performance-improvements-encapsed-strings-optimization.html but after a second look, it looks like a micro-optimisation. I'll switch to sprintf.

stof · 2017-02-01T18:05:36Z

src/Symfony/Component/Asset/Preload/PreloadManagerInterface.php

+     *
+     * @return string|null
+     */
+    public function getLinkValue();


I think getLinkHeader (or buildLinkHeader) might be a better name

stof · 2017-02-01T18:06:20Z

src/Symfony/Component/Asset/Preload/PreloadManagerInterface.php

+     *
+     * @param array $resources
+     */
+    public function setResources(array $resources);


do we have any use case for replacing all resources, except for clearing ? If no, we could simplify the code by allowing only clearing

javiereguiluz · 2017-02-01T18:51:31Z

src/Symfony/Component/Asset/Preload/PreloadManager.php

+            }
+
+            if (!isset($options['as']) || !is_string($options['as'])) {
+                throw new InvalidArgumentException('The "as" option is mandatory and must be a string.');


I have a question about this behavior because I'm not aware of the related specification. If as is mandatory, would it make sense to make it optional and default its value to the appropriate value according to the extension of the asset? For example, if the asset is foo.js then use script as as automatically.

@javiereguiluz according to the spec it's not mandatory.

I was thinking about a guesser, but without inspecting the content of the file it's difficult to do it reliably.

dunglas · 2017-02-01T19:19:21Z

@stof done. I've also remove the PreloadManager::getResources() because it was leaking an internal state and was not used.

dunglas · 2017-02-01T19:21:27Z

By the way, the preload system is now 100% decoupled of the rest of the Asset component. It may be moved to its own component and its own Twig extension but I'm not sure it's worth it (it's only a couple of files).

fabpot · 2017-02-01T19:50:48Z

src/Symfony/Bridge/Twig/Extension/AssetExtension.php


-    public function __construct(Packages $packages)
+    public function __construct(Packages $packages, PreloadManagerInterface $preloadManager = null)


The default PreloadManagerInterface implementation is simple and does not have dependencies, so I suppose it does not make sense to have a proper runtime.

Sorry I don't get what you mean.

Maybe we're just missing some doc about twig runtimes for extensions? :)

stof · 2017-02-02T09:27:49Z

src/Symfony/Component/Asset/EventListener/PreloadListener.php

+{
+    private $preloadManager;
+
+    public function __construct(PreloadManager $preloadManager)


the typehint must use the interface

stof · 2017-02-02T09:29:02Z

src/Symfony/Component/Asset/EventListener/PreloadListener.php

+        }
+
+        if ($value = $this->preloadManager->buildLinkValue()) {
+            $event->getResponse()->headers->set('Link', $value);


you must pass false as third argument. You don't want to replace the existing Link headers which might exist for other purposes

stof · 2017-02-02T09:30:45Z

src/Symfony/Component/Asset/Tests/EventListener/PreloadListenerTest.php

+        $this->assertInstanceOf(EventSubscriberInterface::class, $subscriber);
+        $this->assertEquals('</foo>; rel=preload', $response->headers->get('Link'));
+        $this->assertNull($manager->buildLinkValue());
+    }


please add a test where the response already has another Link header previously

dunglas · 2017-02-03T08:01:32Z

Should be all good now.

Status: needs review

xabbuh · 2017-02-06T13:44:21Z

General question: Do we want to merge the PR as long as the spec document is in draft state?

dunglas · 2017-02-06T14:10:41Z

@xabbuh it's already broadly used in the wild: Chrome and Opera support this feature, Firefox and Webkit are implementing it, Edge is considering implementing it (https://developer.microsoft.com/en-us/microsoft-edge/platform/status/preload/) and - more interestingly - CloudFlare already supports the transparent conversion of preload links to HTTP/2 pushes so there are immediate benefits for any client supporting HTTP/2 (80% of all modern browsers) if the website uses CloudFlare.

Maybe can we mark it as @experimental in the (unlikely) eventuality that the spec change but I think we should merge it as soon as possible to allow our users to benefit of the already existing performance boost.

dunglas · 2017-02-07T07:43:58Z

@xabbuh btw it's an hot topic, the @GoogleChrome team just released a Webpack plugin that looks very similar: https://github.com/googlechrome/preload-webpack-plugin (but our solution is better because CloudFlare supports only HTTP headers, not HTML links yet).

They also support prefetch while we just support preload for now. What do you think of adding a new prefetch Twig tag?

xabbuh · 2017-02-07T14:47:34Z

src/Symfony/Component/Asset/Preload/PreloadManager.php

+ *
+ * @author Kévin Dunglas <dunglas@gmail.com>
+ */
+class PreloadManager implements PreloadManagerInterface


could be final

would forbid decoration for no reason imho - see comments on the interface: building a profiler panel could require decorating this class

I meant decoration by inheritance of course.
My root issue is that final here would be only forbidding use cases while providing no benefit for us.
"final" should only used on method/classes that are not bound by any contract - like data objects.
When an interface covers the said methods, the contract is already enforced and inheritance should not be prevented.

My root issue is that final here would be only forbidding use cases while providing no benefit for us.

There are 2 benefits:

changing the methods is less risky, see for example: [Translation] small performance improvement for MessageCatalogue::replace #11708, it was rejected just because of potential BC breaking;

discourage "decoration by inheritance" when you have an interface with the same methods: in most cases there is no sane reason to do that, it is probably design mistake.

When an interface covers the said methods, the contract is already enforced and inheritance should not be prevented.

I don't see connection between contract enforcing and "inheritance should not be prevented". I'd say the opposite. See similar opinion, for example: http://ocramius.github.io/blog/when-to-declare-classes-final/ /cc @Ocramius

fabpot · 2017-02-19T22:57:30Z

Thank you @dunglas.

…/2 push (dunglas) This PR was squashed before being merged into the 3.3-dev branch (closes #21478). Discussion ---------- [Asset] Add support for preloading with links and HTTP/2 push | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | n/a | License | MIT | Doc PR | todo Allows compatible clients to preload mandatory assets like scripts, stylesheets or images according to [the "preload" working draft of the W3C](https://www.w3.org/TR/preload/). Thanks to this PR, Symfony will automatically adds `Link` HTTP headers with a `preload` relation for mandatory assets. If an intermediate proxy supports HTTP/2 push, it will convert preload headers. For instance [Cloudflare supports this feature](https://blog.cloudflare.com/using-http-2-server-push-with-php/). It dramatically increases pages speed and make the web greener because only one TCP connection is used to fetch all mandatory assets (decrease servers and devices loads, improve battery lives). Usage: Updated version: ```html <html> <body> Hello <script src="{{ preload(asset('/scripts/foo.js'), 'script') }}"></script> </body> </html> ``` ~~First proposal:~~ ```html <html> <body> Hello <script src="{{ preloaded_asset('/scripts/foo.js', 'script') }}"></script> </body> </html> ``` - [x] Add tests Commits ------- 7bab217 [Asset] Add support for preloading with links and HTTP/2 push

fabpot · 2017-02-19T22:57:59Z

@dunglas Can you submit a PR on the docs?

javiereguiluz · 2017-03-06T16:29:06Z

I know this is already merged but ... could we rename the "negative argument" nopush (with false as default) for a "positive argument" called push (with true as default) in the preload() function? Negative variables are always so confusing!

{# before #}
{{ preload('...', { as: 'style', nopush: true }) }}

{# after #}
{{ preload('...', { as: 'style', push: false }) }}

robfrawley · 2017-03-06T16:35:13Z

While I follow your logic (and generally agree) @javiereguiluz, I think this is named as such to align directly with the generated code and referenced standard, which will only ever have a reference to nopush or nothing at all, ie:

Link: </app/style.css>; rel=preload; as=style; nopush
Link: </app/script.js>; rel=preload; as=script

In the context of this PR I think it should remain as-is.

dunglas · 2017-03-06T16:42:33Z

@robfrawley it was exactly my rationale :)
Being as close as possible of the standard is better here.

@evert

This PR was squashed before being merged into the 3.3-dev branch (closes #22273). Discussion ---------- Add a new Link component | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? |no  | BC breaks? | no | Deprecations? | no  | Tests pass? | yes | Fixed tickets | n/a | License | MIT | Doc PR | todo This a proposal to extract HTTP preloading features introduced in #21478 in their own component. There are some good reasons to do it: * HTTP preloading is not (only) about assets: this standalone component could be very useful to replace resources embedding in APIs by HTTP/2 pushes like described in [this article](https://evertpot.com/rest-embedding-hal-http2/) by @evert. In such case, there is no reason to carry the whole asset component for an API. * There is no dependency nor relation at all between the code of the asset compnent and the one I've added for Preloading features * It makes the code cleaner (no more optional dependency in the `Asset` Twig extension) This component would also better fit in HttpFoundation than in Asset. But there is no dependency between it and HttpFoundation and it can easily be used with PSR-7 too, so IMO it better belongs in a standalone component. Btw, ~~~I plan to add support for prefetching to this component. Except a PR soon.~~~ Prefetching and prerendering support added in this PR. ping @symfony/deciders Commits ------- 053de25 Add a new Link component

@nicolas-grekas

This PR was merged into the 3.4 branch. Discussion ---------- Add docs for the WebLink component The WebLink component is available since Symfony 3.3, but I never took the time to add the docs (however, a blog post explaining how to use it was available). This documentation is based on https://dunglas.fr/2017/10/symfony-4-http2-push-and-preloading/. If necessary, I can grant any copyright regarding this post to the Symfony project. symfony/symfony#21478 symfony/symfony#22273 Closes #7515. Commits ------- 91ee3bc Fix RST ea7b3da @nicolas-grekas' review 38fda88 fix build e12e776 RST 088690f Fix link e3d4036 RST 178821e refactor 9f4ae9b fix typo 6beb4eb Add docs for the WebLink component

carsonbot added Status: Needs Review Asset Feature labels Jan 31, 2017

nicolas-grekas added this to the 3.3 milestone Jan 31, 2017

nicolas-grekas added Status: Needs Work and removed Status: Needs Review labels Jan 31, 2017

nicolas-grekas changed the title ~~[Asset][WIP] Add support for preloading with links and HTTP/2 push~~ [Asset] Add support for preloading with links and HTTP/2 push Jan 31, 2017

lyrixx reviewed Feb 1, 2017

View reviewed changes

stof requested changes Feb 1, 2017

View reviewed changes

dunglas force-pushed the preload branch from 8f1e708 to 34dc9b5 Compare February 1, 2017 17:44

stof reviewed Feb 1, 2017

View reviewed changes

javiereguiluz reviewed Feb 1, 2017

View reviewed changes

fabpot reviewed Feb 1, 2017

View reviewed changes

stof requested changes Feb 2, 2017

View reviewed changes

dunglas force-pushed the preload branch from 348aed1 to 59f3b3e Compare February 3, 2017 08:01

carsonbot added Status: Needs Review and removed Status: Needs Work labels Feb 3, 2017

dunglas force-pushed the preload branch from eba8ffc to 7da5258 Compare February 4, 2017 10:38

dunglas added 13 commits February 7, 2017 11:01

Add tests and nopush support

f920ca4

CS

1de19cc

Fix tests

c5ca4bc

Preload is now a standalone Twig function

639d7d1

Make the preload manager independant of HTTP Foundation

9c11a0f

Fix CS

d39b4f1

Fix listener

581c700

Fix @stof's comments

693107c

Fix CS

2d8d750

Don't replace existing link

0b7fe14

Fix tests

dffb7b6

Some more fixes

18412e5

Remove HttpKernel from suggest

78d7b6a

dunglas force-pushed the preload branch from 7da5258 to 78d7b6a Compare February 7, 2017 10:01

dunglas added 2 commits February 7, 2017 11:02

Remove unseless null

05134b6

Update skip message

a7f77b4

xabbuh reviewed Feb 7, 2017

View reviewed changes

fabpot closed this Feb 19, 2017

xabbuh mentioned this pull request Feb 20, 2017

Add docs for the new WebLink component symfony/symfony-docs#7515

Closed

dunglas mentioned this pull request Apr 4, 2017

Add a new Link component #22273

Closed

fabpot mentioned this pull request May 1, 2017

Release v3.3.0-BETA1 #22603

Merged

dunglas mentioned this pull request Sep 10, 2018

Add docs for the WebLink component symfony/symfony-docs#10309

Merged


		public function __construct(Packages $packages)
		public function __construct(Packages $packages, PreloadManagerInterface $preloadManager = null)

[Asset] Add support for preloading with links and HTTP/2 push #21478

[Asset] Add support for preloading with links and HTTP/2 push #21478

Conversation

dunglas commented Jan 31, 2017 • edited

dunglas commented Jan 31, 2017 • edited

pkruithof commented Feb 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

javiereguiluz commented Feb 1, 2017

dunglas commented Feb 1, 2017 • edited

javiereguiluz commented Feb 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dunglas commented Feb 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dunglas Feb 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dunglas Feb 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dunglas Feb 1, 2017 • edited

Choose a reason for hiding this comment

dunglas commented Feb 1, 2017

dunglas commented Feb 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dunglas commented Feb 3, 2017

xabbuh commented Feb 6, 2017

dunglas commented Feb 6, 2017 • edited

dunglas commented Feb 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabpot commented Feb 19, 2017

fabpot commented Feb 19, 2017

javiereguiluz commented Mar 6, 2017

robfrawley commented Mar 6, 2017 • edited

dunglas commented Mar 6, 2017

dunglas commented Jan 31, 2017 •

edited

dunglas commented Jan 31, 2017 •

edited

dunglas commented Feb 1, 2017 •

edited

dunglas Feb 1, 2017 •

edited

dunglas Feb 1, 2017 •

edited

dunglas Feb 1, 2017 •

edited

dunglas commented Feb 1, 2017 •

edited

dunglas commented Feb 6, 2017 •

edited

robfrawley commented Mar 6, 2017 •

edited