add support for lazy loading injected dependencies via a proxy #5012

Closed
lsmith77 opened this Issue Jul 22, 2012 · 32 comments
@lsmith77

right now there are situations where people inject the DIC just because they don't want to incur the overhead of explicit injection of optional parameters. for this case we provide a way to automatically generate a proxy object instead similar to how Doctrine proxy objects work.

@lsmith77

also just a crazy idea, not sure if its possible or even wanted .. but maybe optionally the proxy could even be aware of scope changes at which point if the service was loaded it would be reset in the proxy.

this way one could potentially inject the request service but always get the correct instance for the current sub request.

@Richtermeister

I'm actually doing Container-injection all over the place, and yes, that would be a great feature, if possible.

@jakzal
Symfony member

@lsmith77 how would you approach this?

Proxy class would have to extend the service class in order to make it possible to pass it around instead of a real service. This would cause problems if there were any initializations in the constructor. Effectively we would be creating the actual service anyway.

Ideally proxy class should implement the same interface as a service. I'm afraid this would be to big limitation since not always we're in a control of the service code (think of third party classes).

One approach would be to store a service in a property of a proxy and create it first time it's actually used. Proxy would still have to implement the same interface as the service (we could probably inspect it) or extend it.

@lsmith77
@ebuildy

Absolutly right ! Currently tons of services class are created but never used,

My approach will be to parse the container XML (inside cache folder), get each class and create the proxy using Reflection tools. Or use getter methods instead constructor parameters in the service class so the depencies will be create only on demand.

Is it something planed ?

@lsmith77
@lsmith77

i of course meant @ocramius

@Ocramius

@lsmith77 I do not think it will land in Doctrine 2.3 though.

@lsmith77

yeah .. well this feature will not land in Symfony 2.1 either ..

@andrerom

+1, would avoid a lot of container awareness that is starting to infest quite some code.

@richardmiller

I'm not sure that the proxy classes should be used as a matter of course but that you can mark certain services to be proxied. This can then be used when something is injected somewhere every time but only used on some requests which is the situation where the controller is injected most often currently. Only the top level service would need to be proxied and if it is required it would be safe to set it up fully presumably. Surely proxying everything would be worse for performance if classes that are not heavyweight to instantiate are being proxied when they will be used on most requests.

@ebuildy

It depends, proxy will stop huge class creation due to cascading effect (especially for SF components), in other hand, inject container in all services its useless and will make SF2 like codeIgniter (with much better perf for CI of course ;-)), what do you propose ?

@schmittjoh

There is a discussion on the mailing list which might help here:
https://groups.google.com/forum/?fromgroups=#!topic/symfony-devs/_BSturC3diA

@richardmiller

Not injecting the container at all if possible but only using proxies strategically to avoid unnecessary object creation - I just think that using proxies for everything and then swapping it out for the real service straight away will be a bad idea

@lsmith77

i think we need a syntax for this ..
not sure if we can just assume that we should turn any optional dependency into a proxy in case the service exists (for yaml for xml we can introduce a new specific syntax).

@richardmiller

I agree I think there needs to be a way of specifically marking a service as one to be proxied.

@andrerom

Don't know about the rest of you, but in me/our (eZ) case we mostly need this in cases where we need a list of services (that implements a interface) on demand (to avoid mentioned cascading effect of dependencies of dependencies being loaded, loading the whole system).

<verbose>
Before we switched to Symfony Container we used closures to wrap around the container in these cases which would then load the service when called (also supporting callsback so you could define "::helloWorld", and then doing $service() would load and then do the call).
But this is a weakly typed solution, and means code needs to be adapted for it, so not a good one.
</verbose>

If the container would be cable of generating a ArrayObject that does lazy loading in the background, and supports being able to access the services by key, then at least most of our use cases will be covered, and doc hinting type hinting is possible when wrapped inside a thin factory.

Factory without container knowledge example, fully unit testable w/o container:

/**
* @param \ServiceInterface[] $listOfServices A hash of services where key is the identifier of the service, accepts array or ArrayAccess
*/
function __construct( $listOfServices ){ .. }

/**
* @throws InvalidArgument When Service with $identifier is not found
*
* @return \ServiceInterface
*/
function getByIdentifier( $identifier )
{
    //(...)
    return $this->services[$identifier];
}

Configuration could for instance be done by means of @:<tagName> to get all services by given tag in the form of a lazy loaded ArrayObject, where key is the name of the service or a identifier defined in service configuration.

@Richtermeister

Hey Andre,

in your case, if you just use a factory class you already have lazyloading. You just wouldn't store the whole instantiated service in the array, but only the service ids, and then also provide the factory with the container, and voila, look up any service and it's lazyloaded.

Overall though I love the idea of adding a lazyload parameter to service definitions, so this can be applied selectively.
Maybe this should actually be part of the way that arguments are being passed, because I may want to skip the proxy generation when I access a service directly, but want to proxy it when I inject it into another service that may not use it. Maybe just like we use the @? to handle optional dependencies, we could use some other character (@~ or whatever) to indicate proxies..

@andrerom

factory class you already have lazyloading

yes, but in this case the factory doesn't need to have knowledge about the container, hence de coupled from the DIC used. But as this is a separate enhancement request effectively, I'll find time to create a issue for this.

we could use some other character (@~ or whatever) to indicate proxies..

+1 on @~ or at least a way to separate proxies from instances in config, proxies add overhead and it should be possible to select it on a case by case basis.

@richardmiller

-1 on @~ I think it should be an explicit setting, something like use-proxy="true", lazy-load="true" etc.

@lsmith77

@andrerom i also think that the proxying should be configurable on a per service dependency definition bases. ie. a service can be used with and without proxying.

for those cases when someone always wants the service to be proxied, then we can offer an abstract factory service definition.

@stof
Symfony member

@richardmiller for the @~, we are talking about a syntax for YAML, which does not have attribute. XML does not use @ at all to reference services.

@richardmiller

@stof - fair point, I was thinking of something like use_proxy: true but I guess being explicit isn't a priority when using Yaml.

Thinking about it though this is not just about the syntax, its whether it is specified as part of the service definition or specified when it is being injected into another service.

@stof
Symfony member

@richardmiller You cannot put use_proxy: true in the argument list.

And I don't think the proxy should be declared on the service itself but in places where it is injected: the guy writing a service depending on it can know whether it makes sense to proxy the service (optional dependency used only in some cases) or if it would only add overhead (required dependencies used whenever you use the service). Doing it in the service definition means you are making assumptions about the way your service is used.

@richardmiller

@stof yep, I understand about the need for the syntax in the argument list. I agree that by putting the decision at the point where the service is used it the decision can be based on whether the service will only be used in some circumstances. Marking the service itself would allow highlighting services that are heavy to construct and where proxying it will actually gain something - this does sound like more of a documentation issue though since it could just be advisory.

@lsmith77
@Ocramius

@cordoval keep an eye on this, since it's the main subject of our hacking this Saturday.

@Ocramius

@lsmith77 I've implemented it for ZF2 at zendframework/zendframework#2995 (for those curious about the approach). Will work on a symfony version of this shortly

@henrikbjorn

For the syntax what about <tag-name-here> (taken from the array<SomeClass> which some documentation engines uses.

@lsmith77

this would also solve some issues around scoping of services that need the request, but that can also be instantiated outside of a request lunetics/LocaleBundle#43 where currently its necessary to then inject the DIC.

@fabpot fabpot added a commit that referenced this issue Mar 23, 2013
@fabpot fabpot merged branch fabpot/contagious-services (PR #7007)
This PR was merged into the master branch.

Discussion
----------

[2.3] [WIP] Synchronized services...

| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #5300, #6756
| License       | MIT
| Doc PR        | symfony/symfony-docs#2343

Todo:

 - [x] update documentation
 - [x] find a better name than contagious (synchronized)?

refs #6932, refs #5012

This PR is a proof of concept that tries to find a solution for some problems we have with scopes and services depending on scoped services (mostly the request service in Symfony).

Basically, whenever you want to inject the Request into a service, you have two possibilities:

 * put your own service into the request scope (a new service will be created whenever a sub-request is run, and the service is not available outside the request scope);

 * set the request service reference as non-strict (your service is always available but the request you have depends on when the service is created the first time).

This PR addresses this issue by allowing to use the second option but you service still always has the right Request service (see below for a longer explanation on how it works).

There is another issue that this PR fixes: edge cases and weird behaviors. There are several bug reports about some weird behaviors, and most of the time, this is related to the sub-requests. That's because the Request is injected into several Symfony objects without being updated correctly when leaving the request scope. Let me explain that: when a listener for instance needs the Request object, it can listen to the `kernel.request` event and store the request somewhere. So, whenever you enter a sub-request, the listener will get the new one. But when the sub-request ends, the listener has no way to know that it needs to reset the request to the master one. In practice, that's not really an issue, but let me show you an example of this issue in practice:

 * You have a controller that is called with the English locale;
 * The controller (probably via a template) renders a sub-request that uses the French locale;
 *  After the rendering, and from the controller, you try to generate a URL. Which locale the router will use? Yes, the French locale, which is wrong.

To fix these issues, this PR introduces a new notion in the DIC: synchronized services. When a service is marked as synchronized, all method calls involving this service will be called each time this service is set. When in a scope, methods are also called to restore the previous version of the service when the scope leaves.

If you have a look at the router or the locale listener, you will see that there is now a `setRequest` method that will called whenever the request service changes (because the `Container::set()` method is called or because the service is changed by a scope change).

Commits
-------

17269e1 [DependencyInjection] fixed management of scoped services with an invalid behavior set to null
bb83b3e [HttpKernel] added a safeguard for when a fragment is rendered outside the context of a master request
5d7b835 [FrameworkBundle] added some functional tests
ff9d688 fixed Request management for FragmentHandler
1b98ad3 fixed Request management for LocaleListener
a7b2b7e fixed Request management for RequestListener
0892135 [HttpKernel] ensured that the Request is null when outside of the Request scope
2ffcfb9 [FrameworkBundle] made the Request service synchronized
ec1e7ca [DependencyInjection] added a way to automatically update scoped services
74f96bf
@Ocramius Ocramius referenced this issue Mar 30, 2013
Closed

Lazy services - service proxies #7527

8 of 8 tasks complete
@fabpot
Symfony member

Closing as the discussion should now happen on #7527

@fabpot fabpot closed this Apr 21, 2013
@fabpot fabpot added a commit that referenced this issue May 6, 2013
@fabpot fabpot merged branch Ocramius/feature/proxy-manager-bridge (PR #7890)
This PR was squashed before being merged into the master branch (closes #7890).

Discussion
----------

ProxyManager Bridge

As of @beberlei's suggestion, I re-implemented #7527 as a new bridge to avoid possible hidden dependencies.

Everything is like #7527 except that the new namespace (and possibly package/subtree split) `Symfony\Bridge\ProxyManager` is introduced

| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | yes
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #6140 (supersedes) #5012 #6102 (maybe) #7527 (supersedes)
| License       | MIT (attached code) - BSD-3-Clause (transitive dependency)
| Doc PR        | Please pester me to death so I do it

This PR introduces lazy services along the lines of zendframework/zendframework#4146

It introduces an **OPTIONAL** dependency to [ProxyManager](https://github.com/Ocramius/ProxyManager) and transitively to [`"zendframework/zend-code": "2.*"`](https://github.com/zendframework/zf2/tree/master/library/Zend/Code).

## Lazy services: why? A comprehensive example

For those who don't know what this is about, here's an example.

Assuming you have a service class like following:

```php
class MySuperSlowClass
{
    public function __construct()
    {
        // inject large object graph or do heavy computation
        sleep(10);
    }

    public function doFoo()
    {
        echo 'Foo!';
    }
}
```

The DIC will hang for 10 seconds when calling:

```php
$container->get('my_super_slow_class');
```

With this PR, this can be avoided, and the following call will return a proxy immediately.

```php
$container->getDefinitions('my_super_slow_class')->setLazy(true);
$service = $container->get('my_super_slow_class');
```

The 10 seconds wait time will be delayed until the object is actually used:

```php
$service->doFoo(); // wait 10 seconds, then 'Foo!'
```

A more extensive description of the functionality can be found [here](https://github.com/Ocramius/ProxyManager/blob/master/docs/lazy-loading-value-holder.md).

## When do we need it?

Lazy services can be used to optimize the dependency graph in cases like:

 * Webservice endpoints
 * Db connections
 * Objects that cause I/O in general
 * Large dependency graphs that are not always used

This could also help in reducing excessive service location usage as I've explained [here](http://ocramius.github.com/blog/zf2-and-symfony-service-proxies-with-doctrine-proxies/).

## Implementation quirks of this PR

There's a couple of quirks in the implementation:

 * `Symfony\Component\DependencyInjection\CompilerBuilder#createService` is now public because of the limitations of PHP 5.3
 * `Symfony\Component\DependencyInjection\Dumper\PhpDumper` now with extra mess!
 * The proxies are dumped at the end of compiled containers, therefore the container class is not PSR compliant anymore

Commits
-------

78e3710 ProxyManager Bridge
dfd605f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment