Going Forward?

[XedinUnknown]

Hi!

I have been wanting to consolidate effort and give this project another shot, because I believe that the ability to configure DI containers in an interoperable way is very useful. In fact, me and my colleague have created a modularity standard based on this ability, and on this experimental standard. It is now being used in multiple commercial project, proving its viability.

Please take a look at my article Cross-Platform Modularity in PHP, where I explain the advantages of such a modularity system, and how we have managed to achieve it. I wrote this article in an attempt to make a compelling case to support the Service Provider standard.

If the Service Provider standard doesn't make it into a PSR, it would still be great to release it, and I happen to have an organisation - @Dhii - that was created to do exactly such things. I'd be happy to maintain this standard, but of course I would much prefer if it became a PSR, or at least got official support from FIG.

So, what is the way to move forward from here? I am a member of the original workgroup tasked with devising this standard, but I have been neglecting my duties - and I'm sorry. Meanwhile this project hasn't received any traction in the recent years. How can we bring it back to life again? @moufmouf, could you kindly advise? I am also interested to hear the opinions of other workgroup members, but not sure what the best way to involve them here is.

[bnf]

Thank you for bringing this up, I want to chime in.
The ideas of this proposal are great and simple and integrate nicely into projects that adhere to PSR-specs. We have therefore successfully included the current state of the ServiceProvider proposal into TYPO3.CMS v10 [1] (for internal use only at the moment) and actually combined it with Symfony DI[2].

We look very much forward to depenend on an official ServiceProvider PSR to provide it as public API in TYPO3. An official PSR would therefore be much appreciated.

Would be great to see the TYPO3 development lead @bmack as voting member (as for the PSR-14 where he participated as well).
For reference he asked about the state in january on the PHP-FIG mailing list: https://groups.google.com/forum/#!topic/php-fig/LJAC-Xn5APo
David expressed that he is still interested in this topic, but it's hard to find a team of people.
@moufmouf We as TYPO3 team would be willing to help with the proposal.

[1] TYPO3/typo3@0cf8053
[2] Based on the ideas of https://github.com/thecodingmachine/service-provider-bridge-bundle

[XedinUnknown]

I think we should start by getting the original workgroup together. I would like to know what they think. There was a lot of time to experiment with this package, and many people could have interesting uses. Thing is, I don't know where the original list is. It's impossible to find anything in Google Groups. I was told once that it's the least bad tool for the job, but I'm fairly sure that Slack could do better. Is there a FIG Slack? :D

[bmack]

@XedinUnknown yes there is, however I'm not sure how to join the Slack - we should ping one of the secretaries or ask on the mailing lists.

[shochdoerfer]

I hope it's fine to share it here, the original working group was:

• Paul Dragoonis (PPI framework)
• Stephan Hochdörfer (Disco)
• Miguel Muscat (dhi/di)
• Matthieu Napoli (PHP-DI / PSR-11 co-editor)
• David Négrier (me, Mouf, PSR-11 co-editor)
• Anton Ukhanev (dhi/di)
• Matthew Weier O'Phinney (Zend framework / PSR-11 sponsor)

I'd absolutely love to drive this forward, even though for the past 3 years I had no idea how. I wanted to find a low-code approach compared to have we already have, but I just could not find a good idea on how to do it.

Ideally, we also could get the well-known frameworks in the game, to come up with something that is "adoptable" broadly.

[samdark]

We have something similar in Yii 3 so I'm interested to join the effort.

[XedinUnknown]

Thanks a lot! I think it would be useful t tag all of them here: @dragoonis, @shochdoerfer, @mecha, @mnapoli, @moufmouf, @XedinUnknown, @weierophinney.

When we decide which people will be working on this project, it would be great if we could put the workgroup members into a GH team. Like this, everyone always knows who is who, and the workgroup can be referenced as a whole. Permissions can also be managed for the whole workgroup.

[weierophinney]

I saw the post in the php-fig group last night, and had a reminder to reply... and now you've tagged me!

Yes, I am definitely interested in working on this!

In Laminas (formerly ZF) and Mezzio (formerly Expressive), we've been introducing a concept of "config providers" that provide a combination of application configuration and dependency configuration. We've actually been able to use the former to seed more than a half-dozen DI containers, which has allowed users to mix-and-match their DI container of choice with Mezzio applications. (We still have a few areas to resolve, though, primarily around our "plugin manager" implementation in laminas-servicemanager, but that largely comes down to cutting a new major version that targets PSR-11 specifically).

Count me in!

[mecha]

Count me in too

[moufmouf]

Hey @XedinUnknown !

Thanks for restarting this!

I have been using this standard extensively too.

In my experience, there are 3 things missing:

1. The ability to manage (ordered) list of services
It is currently possible using extensions (but very hard) to manage lists of services (where a service providers adds a service to a list of service) => think: a service provider declares a middleware and adds it to the list of all middlewares.
I think we need a native way in the standard to have a kind of "tags".
I have discussed this already with others (like @mnapoli ) and I understand this is controversial. I still think that this is a very important feature.

2. The ability to create services dynamically based on configuration
Currently, the service-provider standard mixes configuration and services in a single ContainerInterface. But in typical applications, configuration is available first, and services are available when the container is "frozen/compiled".

As a consequence to this choice we made, there is no way for a service provider to provide a dynamic number of services based on configuration. Let's take an example:

Let's imagine you have a configuration defining several databases URL:

dbs:
 db1: mysql://root@localhost 
 db2: mysql://root:password@192.168.0.2

Currently, we cannot write a ServiceProvider that will offer a "db1" and a "db2" service based on the configuration above. This is a limitation I would like to overcome.

3. The ability to declare services in "loops"

You cannot easily declare services in "loops": a service A references a service B that references service A. Even if you use setter injection, this is impossible with ServiceProviderInterface (because it relies on factories that will go into loop).
Most of the time, having loops is a design smell so I'm fine if we cannot solve this, but to be clear, I have had the issue twice is 3 years, and it was legitimate.

Finally, I'd like to add I have been talking with framework developers in the past years. In particular, I talked about it to Nicolas Grekas from Symfony. One of its issues was that we could not ensure type safety. A service provider should in his opinion expose the type of the services it contains. Quite honestly, this is possible in container-interop/service-providers if you use reflection on the factories returned by "getServices".

He actually went on to implement a ServiceProviderInterface in Symfony that is completely different from what we have here (the scope is different too!) Take a look: https://github.com/symfony/symfony/blob/master/src/Symfony/Contracts/Service/ServiceProviderInterface.php

Finally, if we manage to restart the project and get people involved, I see no objections to opening write access to this repository to people working on the project (maybe to the new working group?). I'll obviously wait for the approval of @mnapoli to do this (he is the one who boostraped this project actually!) container-interop is already well identified by many and if we can avoid fragmentation, that's cool. Obviously, if we come up with a completely different idea, we can test it in another project.

[samdark]

IMO current interfaces are very specific and are suitable only for certain type of containers (Symfony, Laravel). See #52 for example.

[mecha]

A lot of valid points are being brought up here and they range widely in use case, usage and expectations.

So perhaps it would be best if we sought a thread-based communication platform, to easily compartmentalize our discussions. Otherwise I fear the discourse may become too chaotic.

I haven't yet used GitHub's "Team" feature, but it seems to be focused on topics and comments. Does anyone know if it's any good?

[XedinUnknown]

@moufmouf, good to see you in this thread! About your points:

1. What exactly do you mean by "manage lists of services? Every service provider is its own list of services. Multiple service providers can be combined into 1, like we do with CompositeCachingServiceProvider. Our module system is based on the simple principle of load order. Every module exposes a service provider. These service providers are then merged together, and used to configure a DI DelegatingContainer, which therefore becomes the single source of truth for all configuration.

2. A container cannot provide a dynamic number of services based on configuration.

   This depends on what you mean by "configuration". A container can provide a dynamic number of services based on the service providers used to configure the container, through which these services are retrieved.

   In our system, we come from the realization that everything accessible via a DI container is configuration. If a DBAL object needs, say, a connection timeout int in seconds, that's configuration. If a DBAL object needs another Socket object, that's configuration too. Something isn't configuration based on whether or not it is a scalar value. Everything that can be used to configure something is configuration; therefore, everything can be configuration. All values of all types have the same equal "rights" within a DI container. From the Service Provider's perspective, this is slightly different, because all definitions are callable, but that is fine, because callable can return scalars. Also, a CompositeContainer can be used to "combine" multiple containers into one, such that the first inner container that has the requested key wins. This means that it's possible to add an override method for services, for example: if a value exists in the DB it is retrieved from there, then falling back to some combined YAML configuration, and finally looking in the service providers. This is very convenient, and everything is compartmentalized, keeping components agnostic of each other.

   Another approach is: nesting. A container is queried by key, and that key can be a delimited list of segments, together representing a path. It is possible to retrieve 'mysql://root@localhost' from your example by querying a container configured with a respective service provider by doing $c->get('dbs/db1'), for example (depending on the convention used in the application). It's also possible to nest containers. If the dbs key contains a container with db1 and db2, then we can retrieve the same value by doing $c->get('dbs')->get('db1'). Now, let's imagine that there is the following configuration:

   {
   	"db_configs": [
   		{
   			"alias": "db1",
   			"host": "localhost",
   			"username": "root",
   			"password": "mypass"
   		},
   		{
   			"alias": "db2",
   			"host": "192.168.0.2",
   			"username": "root",
   			"password": "password"
   		}
   	]
   }

   It is entirely possible to have a container at dbs that would provide connection instances based on the above configuration, automatically, as new connection configuration objects get added, such that if I add this:

   {
   	"db_configs": [
   		{
   			"alias": "db3",
   			"host": "myhost",
   			"username": "root",
   			"password": "mypass"
   		}
   	]
   }

   I would be able to retrieve a connection configured that way by doing $c->get('dbs/db3').

   My point here is: all of these things are possible, but none of them have to be the concern of the Service Provider. The service provider is there to provide services. The way they are accessed etc is not really its concern.

3. Recursive definitions are impossible with Service Provider because they are impossible in practice. If you have two immutable classes that depend on each other, you cannot instantiate them, because they can never satisfy each other's requirements. It is possible to go around this limitation using mutable proxies, which has the benefit of leaving the other parties immutable.

   Consider this scenario, which often takes place in my applications. Services provided by DelegatingContainer need to be cached, but caching is not its concern. This is provided by the CachingContainer decorator, which is ultimately consumed from the outside, like in glue code. But DelegatingContainer will pass itself to its service definitions, not CachingContainer, which means that any services that are retrieved by other services will not get the cached version. DelegatingContainer accepts an optional $parent argument, which can be a container to delegate secondary resolution to, i.e. this is what will be passed to its service definitions. So, DelegatingContainer and CachingContainer depend on each other. The solution is to use ProxyContainer:

   function createAppContainer (ServiceProviderInterface $serviceProvider): ContainerInterface {
   	$proxy = new ProxyContainer();
       $appContainer = new CachingContainer(
           new DelegatingContainer($serviceProvider, $proxy)
       );
       $proxy->setInnerContainer($appContainer);
   
   	return $appContainer;
   }

   I'm sure that a similar concept can be applied to make recursive dependency possible. But again, I don't believe that this is a concern for Service Provider.

---

With regard to type safety. Service definitions can be type-safe, because their return type can be hinted, meaning that it is guaranteed to return the required type. This can be used for static code analysis, making IDE integrations possible. However, I don't see how anything retrieved from ContainerInterface::get() can ever be type-safe, even with generics, because one container will always promise one type, whether it's mixed or int or MyDbalThing. In our modular philosophy, there's always the one container that always serves as a single source of truth, even if there are other containers within it.
So, I don't think it's even theoretically possible to achieve type-safety with ContainerInterface.

That said, a certain degree of safety is achieved by having definitions look like this:

[
    'my_service' => function (ContainerInterface $c): MyService {
        $myThing = $c->get('my_thing');
        assert($myThing instanceof MyThing);
        
        return new MyService($myThing);
    },
]

That way, in dev environments, PHP will let you know if the thing you are retrieving does not match your expectations - before you actually need it somewhere. This also provides hints to clever IDEs, which can sometimes also be achieved with /* @var $myThing MyThing */. The practical runtime utility of this is debatable, because in the above example MyService will declare the type of its dependency to be MyThing anyway, which will cause the code to fail just the same, revealing the problem. It can, however, be useful for static analysis, yet not very straight-forward to analyze. Which is why we went further.

What if, instead of having your service definition depend on just ContainerInterface $container, it declared dependency on its actual requirements, like so:

[
    'my_service' => function (MyThing $myThing): MyService {
        // No need for retrieval and type-checking anymore
        return new MyService($myThing);
    },
]

There would just need to be a way to map the parameter MyThing $myThing to a service. This is done by Factory: a callable class that accepts a list of dependency service names and a callback, and maps those services to the parameters of the latter. Now, if my_service depends on my_thing of type MyThing, it can be declared as so:

[
    'my_service' => new Factory(['my_thing'], function (MyThing $myThing): MyService {
        return new MyService($myThing);
    }),
]

Thus, to all service definitions everything is automatically type-safe, static code analysis is possible, and so are IDE integrations. Moreover, since services now declare dependency on other services, it's possible to build a dependency graph. This is very very good, because modules may have dependencies that are satisfied by other modules. For example, a module that uses an HTTP client may declare dependency on a service http/client, depending just on Psr\Http\Client\ClientInterface, and not providing an actual implementation, leaving that choice to something else - like the application, which can then decide which HTTP client is used throughout. I have found this kind of delegation extremely useful. Now, with the above Factory approach, it's possible to automatically generate a dependency graph, telling the consumer - and the user - which dependencies of a module need to be satisfied in order for it to work, before they are needed. Based on that, other cool things are possible, like automatic suggestions of modules that satisfy those dependencies. See the experimental branch for more helpful service decorators!

The good thing is: because SP currently declares a very open callable type of service, these things are optional. I would contend that they should be used by everyone to make their applications more sane and usable, but right now it's not required. Whether or not to make something like this a requirement remains to be discussed. On one thing, it would allow indexing all modules, like PackAnalyst does (thanks for that!). On the other, there can be many simpler cases where this is not required. I remember the Service Provider standard being kinda that way in the beginning, but it was ruled out as over-complicated. Maybe, it can become an additional standard, on top of Service Provider.

I'd love to know what you guys think about all this!

[XedinUnknown]

With regard to a communications platform that @mecha mentioned, I would love to give a shot to Github Teams, or something else!

[XedinUnknown]

Hey all! Can I do something to help move the discussion along? Maybe explain something better? Or the other way around, summarize the above in simpler words? Show more examples? :)

[shochdoerfer]

Can we split the discussion into multiple issues or move to a different platform? I know we all have a lot of ideas, discussing all of them in this single issue won't scale I think.

[XedinUnknown]

I agree, and that's why I think that a different communication tool would be a good idea. I was hoping that maybe FIG members could point us to a FIG Slack or something. If not, we can just create separate issues for now. But then, what issues do we create? We would need to identify the topics first, and I hope we can do that in this initial discussion here.

[dragoonis]

Hi everyone,

Thanks for the ping @XedinUnknown @shochdoerfer.

We can certainly use a room in the FIG slack to discuss things, however in the interest of transparency for the community then a github issue is pretty decent too (right tool right job).

As it turns out I was looking at ContainerInterop (PSR-11) last night, whilst assisting someone with modernizing their framework to use PSR-11 based tool, and I came across PHP-DI, and PimpleInterop and so forth.

@XedinUnknown The DelegatingContainer / CompositeContainer approach is how I solved it in PPI Framework, when composing multiple DiC's from multiple frameworks/packages/sources at once. So you're probably onto something there :-)

Of the time that's passed from launching PSR-11/ContainerInterop until now, and I'm wondering what the demand is, and what the gaps are when trying to provide a generic container standard that fits the majority of all use cases by PHP projects (yii, zend, laravel, symfony ..etc). @moufmouf expressed a few things missing, but where' the drive for that coming from? I think this is a reasonable question to ask before we go deep into solving something.

From a high-level POV I'd like to know what are the challenges that ContainerInterop current faces when integrating it into projects like those mentioned above.

Thanks. <3

[XedinUnknown]

@dragoonis, thanks for joining!

The first version of our module standard used to use ContainerInterface, and then I used to combine the containers of all modules using a CompositeContainer. But this is something that is much easier with ServiceProvider: it avoids each module having to include their own dependency on yet another PSR-11 implementation, because an ad-hoc ServiceProvider implementation is a piece of cake and doesn't take much space. Of course, a composite container still has its uses, such as for "merging" multiple containers, providing overrides, or hooking a modular system into a bigger system. But now I instead merge service providers, and this is important, because extensions are merged differently to how factories are merged.

I really like the way SP is right now, because I think that currently it addresses all concerns that are related to providing extendable services. I don't think that containers themselves, or how they are consumed, is very much the scope of SP. For example, container cannot be compiled themselves, because their members are not enumerable. But Service Providers are, because multiple service providers can be merged into one, and then combined definitions can be enumerated and cached separately, which will save speed when the definitions have to be resolved by a container. SP also allows extensions, because the callable type of definitions is very open. I wrote about it too in the message above, linking to some extensions that we came up with, that are BC with the original vanilla spec.

[romm]

We can certainly use a room in the FIG slack to discuss things, however in the interest of transparency for the community then a github issue is pretty decent too (right tool right job).

Just my two cents, but I'm interested in the discussion indeed and would really like to be able to read it easily 🙂. I'm currently watching the repo and getting emails when new messages are posted, this is perfect for me.

By the way, thanks a lot to all of you for your work, if I think I can somehow participate in the debate I won't hesitate!

[XedinUnknown]

By the way, another way we could expedite this is by doing a synchronous call with the workgroup members. We're all grown-ups here :) Who wants?

[samdark]

When?

[XedinUnknown]

I'm generally flexible, and in the CEST zone.

[mecha]

CEST zone here too.

[mecha]

BTW @moufmouf

The ability to manage (ordered) list of services

Take a look at our work here, specifically ServiceList and ArrayExtension. The latter lacks specifying exact placement in the list, but with granular service providers and clever sorting thereof, the results become incredibly predictable.

[XedinUnknown]

Hello, friends! 👋 Gentle reminder that we don't want this discussion to go stale again :)

[samdark]

@XedinUnknown as others noted, move it to another medium. Try enabling GitHub discussions, for example.

[samdark]

Also we may have a call but we need agenda for it beforehand.

[XedinUnknown]

Yes, I'm saying this too. But I'm not in charge of this.

[samdark]

Who is?

[XedinUnknown]

The maintainers of this package. @moufmouf, can we use something better than this issue?

[XedinUnknown]

If delays can be avoided this way, I would be happy to create a Dhii channel in Slack/Zulip for this.

[moufmouf]

Hey guys,

Sorry for the delay. Let's face it, I don't have enough time to deal with container-interop/service-provider properly right now.
It is frustrating because I have a ton of ideas to move this forward, but just not enough time (and neither does @mnapoli )

@XedinUnknown , you are putting a lof of energy in this and I realize I have been a bottleneck so far.
I added you as admin of this repo. Do not hesitate to build a team and invite more members.
You can also test Github teams, but let's make sure all the discussions stay open and visible to anyone, as asked by @romm.

Finally, so far, we have been working with @mnapoli on a consensus basis (we try to discuss each issue thoroughly and merge when we all agree). Let's keep it this way! :)

[XedinUnknown]

Oh, wonderful! Thanks, @moufmouf!

[XedinUnknown]

Discussing potential enhancement in #53.

[mindplay-dk]

I have followed up on #53, where you can also find a link to an initial draft PSR document.

I am currently working my way through all the open issues, and will reply to all of them - I will either tag the issues (to indicate that they are still ongoing) or close them and explain why.

We've opened up the Discussions tab, and I've started one discussion there, if anyone wants to participate. I will track these and try to keep things moving.

I've been hard at work this weekend, and it may take me a couple of weeks to get through everything.

After that, I would be very open to organizing a group call, if anyone is still interested in that.

If anyone else is interested in participating, give me a shout here? 🙂

[mindplay-dk]

Since this thread is very old, and isn't really about one specific topic, I'm going to close this - if there's a specific topic you'd like to discuss further, please start a Discussion.

Cheers and merry christmas. 🙂