Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Asset] Add support for preloading with links and HTTP/2 push #21478

Closed
wants to merge 16 commits into from

Conversation

@dunglas
Copy link
Member

commented Jan 31, 2017

Q A
Branch? master
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets n/a
License MIT
Doc PR symfony/symfony-docs#10309

Allows compatible clients to preload mandatory assets like scripts, stylesheets or images according to the "preload" working draft of the W3C.
Thanks to this PR, Symfony will automatically adds Link HTTP headers with a preload relation for mandatory assets. If an intermediate proxy supports HTTP/2 push, it will convert preload headers. For instance Cloudflare supports this feature.

It dramatically increases pages speed and make the web greener because only one TCP connection is used to fetch all mandatory assets (decrease servers and devices loads, improve battery lives).

Usage:

Updated version:

<html>
    <body>
    Hello
    <script src="{{ preload(asset('/scripts/foo.js'), 'script') }}"></script>
    </body>
</html>

First proposal:

<html>
    <body>
    Hello
    <script src="{{ preloaded_asset('/scripts/foo.js', 'script') }}"></script>
    </body>
</html>
  • Add tests
@nicolas-grekas nicolas-grekas added this to the 3.3 milestone Jan 31, 2017
@nicolas-grekas nicolas-grekas changed the title [Asset][WIP] Add support for preloading with links and HTTP/2 push [Asset] Add support for preloading with links and HTTP/2 push Jan 31, 2017
@dunglas

This comment has been minimized.

Copy link
Member Author

commented Jan 31, 2017

Test and nopush support added.

Status: needs review

@pkruithof

This comment has been minimized.

Copy link
Contributor

commented Feb 1, 2017

Very nice! We were about to implement this ourselves.

Why use a different function for this? Would an attribute not be more logical?

{{ asset('/scripts/foo.js', 'script', {preload: true}) }}
*/
class HttpFoundationPreloadManager implements PreloadManagerInterface
{
private $resources = array();

This comment has been minimized.

Copy link
@lyrixx

lyrixx Feb 1, 2017

Member

Can you get rides of this state? Because right now it's a memory leak IIUC.

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 1, 2017

Author Member

No it's not possible because this state gather all assets to add to the link.
However I can add a new method to clear it that will be called in the listener to avoid the memory leak.

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

we indeed need to clear the state. Otherwise, using the same kernel to handle multiple requests would leak resources from the previous request. Your implementation breaks the isolation of request handling

@javiereguiluz

This comment has been minimized.

Copy link
Member

commented Feb 1, 2017

I have the same question as @pkruithof: why adding a new preloaded_asset() function instead of adding config options (globally for asset package config and locally for the asset() function).

@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 1, 2017

@pkruithof @javiereguiluz it was my 1st though, however the current signature is: { asset('/path', 'packageName') }.
Both { asset('/path', null, true, 'script', false) } and { asset('/path', null, {'preload': true, 'nopush': true, 'as': 'script') } look bad.
Changing parameters order isn't possible for BC. It's why I've introduced this new tag.

@javiereguiluz

This comment has been minimized.

Copy link
Member

commented Feb 1, 2017

@dunglas thanks for the explanation. Another question: should we name this function asset_preload() instead?

First, it would match the naming followed by other functions, where the first word is "the common thing" (e.g. render_*(), form_*(), asset_*(), etc.)

Second, it would look better when using composition:

{{ preloaded_asset(asset('/scripts/foo.js'), 'script') }}
{{ asset_preload(asset('/scripts/foo.js'), 'script') }}
*
* @author Kévin Dunglas <dunglas@gmail.com>
*/
class PreloadListener

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

should be an event subscriber IMO, like our other listeners, to embed the knownledge about the event being listened to

}

$url = $this->getUrl($path);
$this->preloadManager->addResource($url, $as, $nopush);

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

should the package really be the part being aware of the PreloadManager ? I don't think so. IMO, it should be done at a higher level in the stack. The preloading part is totally independent from the asset url building anyway

*
* @param Response $response
*/
public function setLinkHeader(Response $response)

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

a more reusable API would be to have a method returning the header instead, without dependency on the Response. The event listener would then handle the binding to HttpFoundation by setting the header (allowing the same manager to be reused by people using a PSR-7 stack instead, without reimplementing all the logic in a different class just because of this method).
Currently, the most important part of the business logic is in this method, which is not part of the interface

*/
class HttpFoundationPreloadManager implements PreloadManagerInterface
{
private $resources = array();

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

we indeed need to clear the state. Otherwise, using the same kernel to handle multiple requests would leak resources from the previous request. Your implementation breaks the isolation of request handling

@dunglas dunglas force-pushed the dunglas:preload branch from 8f1e708 to 34dc9b5 Feb 1, 2017
@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 1, 2017

@stof (thanks!!), @lyrixx and @javiereguiluz comments took into account:

  • The preload feature is now 100% independent of the packages and has a Twig function of its own.
  • Dealing with the HttpFoundation's Response is only done in the listener (the manager can now be reused with any HTTP message implementation)
  • Memory leak fixed

The new syntax:

<html>
    <body>
    Hello
    <script src="{{ preload(asset('/scripts/foo.js'), 'script') }}"></script>
    </body>
</html>
@@ -765,6 +767,15 @@ private function registerAssetsConfiguration(array $config, ContainerBuilder $co
$defaultVersion = $this->createVersion($container, $config['version'], $config['version_format'], '_default');
}
if (class_exists(PreloadManager::class)) {

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

missing class existence resource (or you could just conflict with asset < 3.3, and have all this in the XML file directly)

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 1, 2017

Author Member

I don't get what you mean. But if it's ok to bump the dependency, I'll move this definition to the XML, it's cleaner.

return $package
$package

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

Why editing this code ?

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 1, 2017

Author Member

I've done that in a previous commit and it's no useless but I find the new version cleaner/simpler. I can revert it if you prefer the old one.

This comment has been minimized.

Copy link
@stof

stof Feb 2, 2017

Member

changing this without a need for it makes it harder to merge branches together for no reason

@@ -375,6 +376,16 @@ public function testAssetsDefaultVersionStrategyAsService()
$this->assertEquals('assets.custom_version_strategy', (string) $defaultPackage->getArgument(1));
}
public function testAssetHasPreloadListener()
{
if (!class_exists(PreloadListener::class)) {

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

composer ensures that this is always existing

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 1, 2017

Author Member

Not if a version for symfony/asset lesser than 3.3 has been installed, right?

This comment has been minimized.

Copy link
@stof

stof Feb 2, 2017

Member

it cannot, due to the require-dev constraint

public function onKernelResponse(FilterResponseEvent $event)
{
if ($value = $this->preloadManager->getLinkValue()) {

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

this logic must be done only for the master request.
An asset referenced in a subrequest must be preloaded by the master request, as it is the one being sent to the user, and your code would also use all previous master preloaded asset for the subrequest, thus breaking preloading.

foreach ($this->resources as $uri => $options) {
$part = "<$uri>; rel=preload";
if ('' !== $options['as']) {
$part .= "; as={$options['as']}";

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

We generally use sprintf rather than interpolation (even more when it is not simple interpolation)

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 1, 2017

Author Member

I've done it this way because of https://blog.blackfire.io/php-7-performance-improvements-encapsed-strings-optimization.html but after a second look, it looks like a micro-optimisation. I'll switch to sprintf.

*
* @return string|null
*/
public function getLinkValue();

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

I think getLinkHeader (or buildLinkHeader) might be a better name

*
* @param array $resources
*/
public function setResources(array $resources);

This comment has been minimized.

Copy link
@stof

stof Feb 1, 2017

Member

do we have any use case for replacing all resources, except for clearing ? If no, we could simplify the code by allowing only clearing

}
if (!isset($options['as']) || !is_string($options['as'])) {
throw new InvalidArgumentException('The "as" option is mandatory and must be a string.');

This comment has been minimized.

Copy link
@javiereguiluz

javiereguiluz Feb 1, 2017

Member

I have a question about this behavior because I'm not aware of the related specification. If as is mandatory, would it make sense to make it optional and default its value to the appropriate value according to the extension of the asset? For example, if the asset is foo.js then use script as as automatically.

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 1, 2017

Author Member

@javiereguiluz according to the spec it's not mandatory.

I was thinking about a guesser, but without inspecting the content of the file it's difficult to do it reliably.

@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 1, 2017

@stof done. I've also remove the PreloadManager::getResources() because it was leaking an internal state and was not used.

@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 1, 2017

By the way, the preload system is now 100% decoupled of the rest of the Asset component. It may be moved to its own component and its own Twig extension but I'm not sure it's worth it (it's only a couple of files).

public function __construct(Packages $packages)
public function __construct(Packages $packages, PreloadManagerInterface $preloadManager = null)

This comment has been minimized.

Copy link
@fabpot

fabpot Feb 1, 2017

Member

The default PreloadManagerInterface implementation is simple and does not have dependencies, so I suppose it does not make sense to have a proper runtime.

This comment has been minimized.

Copy link
@dunglas

dunglas Feb 3, 2017

Author Member

Sorry I don't get what you mean.

This comment has been minimized.

Copy link
@HeahDude

HeahDude Mar 7, 2017

Member

Maybe we're just missing some doc about twig runtimes for extensions? :)

{
private $preloadManager;
public function __construct(PreloadManager $preloadManager)

This comment has been minimized.

Copy link
@stof

stof Feb 2, 2017

Member

the typehint must use the interface

}
if ($value = $this->preloadManager->buildLinkValue()) {
$event->getResponse()->headers->set('Link', $value);

This comment has been minimized.

Copy link
@stof

stof Feb 2, 2017

Member

you must pass false as third argument. You don't want to replace the existing Link headers which might exist for other purposes

$this->assertInstanceOf(EventSubscriberInterface::class, $subscriber);
$this->assertEquals('</foo>; rel=preload', $response->headers->get('Link'));
$this->assertNull($manager->buildLinkValue());
}

This comment has been minimized.

Copy link
@stof

stof Feb 2, 2017

Member

please add a test where the response already has another Link header previously

@dunglas dunglas force-pushed the dunglas:preload branch from 348aed1 to 59f3b3e Feb 3, 2017
@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 3, 2017

Should be all good now.

Status: needs review

@dunglas dunglas force-pushed the dunglas:preload branch from eba8ffc to 7da5258 Feb 4, 2017
@xabbuh

This comment has been minimized.

Copy link
Member

commented Feb 6, 2017

General question: Do we want to merge the PR as long as the spec document is in draft state?

@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 6, 2017

@xabbuh it's already broadly used in the wild: Chrome and Opera support this feature, Firefox and Webkit are implementing it, Edge is considering implementing it (https://developer.microsoft.com/en-us/microsoft-edge/platform/status/preload/) and - more interestingly - CloudFlare already supports the transparent conversion of preload links to HTTP/2 pushes so there are immediate benefits for any client supporting HTTP/2 (80% of all modern browsers) if the website uses CloudFlare.

Maybe can we mark it as @experimental in the (unlikely) eventuality that the spec change but I think we should merge it as soon as possible to allow our users to benefit of the already existing performance boost.

@dunglas

This comment has been minimized.

Copy link
Member Author

commented Feb 7, 2017

@xabbuh btw it's an hot topic, the @GoogleChrome team just released a Webpack plugin that looks very similar: https://github.com/googlechrome/preload-webpack-plugin (but our solution is better because CloudFlare supports only HTTP headers, not HTML links yet).

They also support prefetch while we just support preload for now. What do you think of adding a new prefetch Twig tag?

dunglas added 13 commits Jan 31, 2017
CS
@dunglas dunglas force-pushed the dunglas:preload branch from 7da5258 to 78d7b6a Feb 7, 2017
dunglas added 2 commits Feb 7, 2017
*
* @author Kévin Dunglas <dunglas@gmail.com>
*/
class PreloadManager implements PreloadManagerInterface

This comment has been minimized.

Copy link
@xabbuh

xabbuh Feb 7, 2017

Member

could be final

This comment has been minimized.

Copy link
@nicolas-grekas

nicolas-grekas Feb 11, 2017

Member

would forbid decoration for no reason imho - see comments on the interface: building a profiler panel could require decorating this class

This comment has been minimized.

Copy link
@nicolas-grekas

nicolas-grekas Feb 11, 2017

Member

I meant decoration by inheritance of course.
My root issue is that final here would be only forbidding use cases while providing no benefit for us.
"final" should only used on method/classes that are not bound by any contract - like data objects.
When an interface covers the said methods, the contract is already enforced and inheritance should not be prevented.

This comment has been minimized.

Copy link
@unkind

unkind Feb 12, 2017

Contributor

My root issue is that final here would be only forbidding use cases while providing no benefit for us.

There are 2 benefits:

  • changing the methods is less risky, see for example: #11708, it was rejected just because of potential BC breaking;
  • discourage "decoration by inheritance" when you have an interface with the same methods: in most cases there is no sane reason to do that, it is probably design mistake.

When an interface covers the said methods, the contract is already enforced and inheritance should not be prevented.

I don't see connection between contract enforcing and "inheritance should not be prevented". I'd say the opposite. See similar opinion, for example: http://ocramius.github.io/blog/when-to-declare-classes-final/ /cc @Ocramius

@fabpot

This comment has been minimized.

Copy link
Member

commented Feb 19, 2017

Thank you @dunglas.

@fabpot fabpot closed this Feb 19, 2017
fabpot added a commit that referenced this pull request Feb 19, 2017
…/2 push (dunglas)

This PR was squashed before being merged into the 3.3-dev branch (closes #21478).

Discussion
----------

[Asset] Add support for preloading with links and HTTP/2 push

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | n/a
| License       | MIT
| Doc PR        | todo

Allows compatible clients to preload mandatory assets like scripts, stylesheets or images according to [the "preload" working draft of the W3C](https://www.w3.org/TR/preload/).
Thanks to this PR, Symfony will automatically adds `Link` HTTP headers with a `preload` relation for mandatory assets.  If an intermediate proxy supports HTTP/2 push, it will convert preload headers. For instance [Cloudflare supports this feature](https://blog.cloudflare.com/using-http-2-server-push-with-php/).

It dramatically increases pages speed and make the web greener because only one TCP connection is used to fetch all mandatory assets (decrease servers and devices loads, improve battery lives).

Usage:

Updated version:

```html
<html>
    <body>
    Hello
    <script src="{{ preload(asset('/scripts/foo.js'), 'script') }}"></script>
    </body>
</html>
```

~~First proposal:~~

```html
<html>
    <body>
    Hello
    <script src="{{ preloaded_asset('/scripts/foo.js', 'script') }}"></script>
    </body>
</html>
```

- [x] Add tests

Commits
-------

7bab217 [Asset] Add support for preloading with links and HTTP/2 push
@fabpot

This comment has been minimized.

Copy link
Member

commented Feb 19, 2017

@dunglas Can you submit a PR on the docs?

@javiereguiluz

This comment has been minimized.

Copy link
Member

commented Mar 6, 2017

I know this is already merged but ... could we rename the "negative argument" nopush (with false as default) for a "positive argument" called push (with true as default) in the preload() function? Negative variables are always so confusing!

{# before #}
{{ preload('...', { as: 'style', nopush: true }) }}

{# after #}
{{ preload('...', { as: 'style', push: false }) }}
@robfrawley

This comment has been minimized.

Copy link
Contributor

commented Mar 6, 2017

While I follow your logic (and generally agree) @javiereguiluz, I think this is named as such to align directly with the generated code and referenced standard, which will only ever have a reference to nopush or nothing at all, ie:

Link: </app/style.css>; rel=preload; as=style; nopush
Link: </app/script.js>; rel=preload; as=script

In the context of this PR I think it should remain as-is.

@dunglas

This comment has been minimized.

Copy link
Member Author

commented Mar 6, 2017

@robfrawley it was exactly my rationale :)
Being as close as possible of the standard is better here.

@dunglas dunglas referenced this pull request Apr 4, 2017
fabpot added a commit that referenced this pull request Apr 10, 2017
This PR was squashed before being merged into the 3.3-dev branch (closes #22273).

Discussion
----------

Add a new Link component

| Q             | A
| ------------- | ---
| Branch?       | master
| Bug fix?      | no
| New feature?  |no <!-- don't forget updating src/**/CHANGELOG.md files -->
| BC breaks?    | no
| Deprecations? | no <!-- don't forget updating UPGRADE-*.md files -->
| Tests pass?   | yes
| Fixed tickets | n/a
| License       | MIT
| Doc PR        | todo

This a proposal to extract HTTP preloading features introduced in #21478 in their own component.

There are some good reasons to do it:

* HTTP preloading is not (only) about assets: this standalone component could be very useful to replace resources embedding in APIs by HTTP/2 pushes like described in [this article](https://evertpot.com/rest-embedding-hal-http2/) by @evert. In such case, there is no reason to carry the whole asset component for an API.
* There is no dependency nor relation at all between the code of the asset compnent and the one I've added for Preloading features
* It makes the code cleaner (no more optional dependency in the `Asset` Twig extension)

This component would also better fit in HttpFoundation than in Asset. But there is no dependency between it and HttpFoundation and it can easily be used with PSR-7 too, so IMO it better belongs in a standalone component.

Btw, ~~~I plan to add support for prefetching to this component. Except a PR soon.~~~ Prefetching and prerendering support added in this PR.

ping @symfony/deciders

Commits
-------

053de25 Add a new Link component
@fabpot fabpot referenced this pull request May 1, 2017
javiereguiluz added a commit to symfony/symfony-docs that referenced this pull request Oct 30, 2018
This PR was merged into the 3.4 branch.

Discussion
----------

Add docs for the WebLink component

The WebLink component is available since Symfony 3.3, but I never took the time to add the docs (however, a blog post explaining how to use it was available).

This documentation is based on https://dunglas.fr/2017/10/symfony-4-http2-push-and-preloading/.
If necessary, I can grant any copyright regarding this post to the Symfony project.

symfony/symfony#21478
symfony/symfony#22273
Closes #7515.

Commits
-------

91ee3bc Fix RST
ea7b3da @nicolas-grekas' review
38fda88 fix build
e12e776 RST
088690f Fix link
e3d4036 RST
178821e refactor
9f4ae9b fix typo
6beb4eb Add docs for the WebLink component
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.