Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re/forming a Working Group #71

Open
mindplay-dk opened this issue Jan 14, 2024 · 35 comments
Open

Re/forming a Working Group #71

mindplay-dk opened this issue Jan 14, 2024 · 35 comments

Comments

@mindplay-dk
Copy link
Collaborator

Issue description

hey folks,

as you may have seen, I've been trying to revive the Service Providers PSR proposal.

I have thoroughly reviewed past discussions, issues, and contributions related to the Service Providers PSR proposal. I've been thinking about this project daily for about two months now. A draft PSR and meta document have been prepared based on the valuable insights gathered from the community over the years. The project is alive and well in my head again, if perhaps currently just in my head. 😅

I'm reaching out to you, the original participants who were actively involved in the Service Providers PSR discussions to understand their current level of interest and participation. if I missed anyone, please give them a mention.

If you were part of the original working group and are still interested in contributing, I would love to hear from you.
Reply to this message, let me know if you are willing to actively participate in the revival of this PSR.

I don't know if we were ever formally a Working Group in PSR terms? either way, members of this Github project have been dormant for so long, I would suggest the next goal should be to formally re-form a Working Group dedicated to the development and refinement of the PSR. This re-formed working group would actively contribute to moving the proposal through the stages of the PSR workflow - starting with the draft stage, which I'd say we're just about ready for now.

I'd be willing to take on the role of "sponsor", facilitating the progress of the PSR proposal through the workflow. I've contributed, but never formally sponsored a PSR - but to my understanding, the sponsor ensures coordination, calls for votes at appropriate stages, and tries to follow the PHP-FIG procedures, etc. I can do that, if you'd like me to?

as said, I'm not aware of any formal Working Group formation ever taking place? but I don't want to step on anybody's toes! if one of you is/was already the "sponsor" and still wants to be, I am completely fine with that as well! I'm not trying to bud in and take the lead on this, unless you want me to. I'm happy to help in whatever way I can. 🙂

but I think, first off, let's reaffirm who wants to be members of the Working Group? If you are still interested in contributing, please reply and let me know - let's put this information in the README and make it "official". If you don't reply, I think it's reasonable we assume you're no longer interested.

After that, if no one objects, I'd like to do an open call for more members to join.

I would love to have your insights and contributions for this effort, but I don't want to presume - so rather than offering an opt-out, I'm calling for you to opt back in. It's been 7 years - no saying whether any of your are still interested or have the time. 🙂

CC in no particular order, just some folks who contributed more recently, plus everyone currently a member of the Github project: @jeremeamia @Ocramius @mnapoli @moufmouf @prisis @Biont @gbprod @XedinUnknown

cheers 🙂

@Ocramius
Copy link
Member

Unable to follow this through FIG's PSR process, mostly because it drains my will to exist.

Ping me on actual interface code reviews though: gladly providing my input there 👍

@mnapoli
Copy link
Member

mnapoli commented Jan 14, 2024

My personal stance is:

  • I'd love for this to happen
  • but this should not happen unless Laravel and Symfony participate (to ensure the PSR gets adopted)

That's why at this point I wouldn't support an effort without them. Happy to revisit if the situation changes.

@shochdoerfer
Copy link

I currently don't have the mental capacity to drive this forward in a way that I think is suitable for the broader PHP ecosystem.

@moufmouf
Copy link
Contributor

Hey @mindplay-dk ,

First of all, thanks for reviving this. I still believe this is the single most useful thing that could happen to the PHP ecosystem.

When I last tried to advance this PSR, both Laravel and Symfony were reluctant (actually, reluctant to any PSR).
On the Laravel side particularly, it is really hard to have any kind of feedback. On the Symfony side, I've had many discussions with Nicolas Grekas at the time so you have better chances there, but their DI container is also the most evolved and the hardest to work with. Anyway, the PSR should absolutely be tailored to work properly with both, and you should not expect them to be too proactive. I also remember Nicolas Grekas saying something about standardizing factories, so that's why I reacted positively to your suggested idea of simply tagging factories with a "#[Factory]" attribute.

As for me, I'm really lacking the time to get involved right now. I need to dedicate 100% of my time to WorkAdventure, which is mostly written in Typescript. As much as I would like this to appear, I don't have the time to participate actively in a working group right now. I'll be happy, just like @Ocramius to do some code reviews and give some feedback though!

@mindplay-dk
Copy link
Collaborator Author

So what you're saying is, the PSR can't happen without Laravel and Symfony - but don't expect Laravel or Symfony to participate, in fact, expect them to be opposed?

I mean, yeah, that does kinda sound like Mission Impossible or Catch 22, doesn't it. 😅

@mnapoli
Copy link
Member

mnapoli commented Jan 15, 2024

the PSR can't happen without Laravel and Symfony - but don't expect Laravel or Symfony to participate

To be fair, we kind of did that with PSR-11. And most of the discussion was IIRC about an exception or two. Both frameworks could implement the interface on day 1 without any impact.

Our initial work with this service-provider proposal was in the same spirit.

So what I said was kind of wrong:

but this should not happen unless Laravel and Symfony participate

It's more that it shouldn't happen (IMO) without (guaranteed) Laravel and Symfony support.

If they won't do the work (which is perfectly fine), you (I mean the person carrying this PSR) would have to do the work. I think @moufmouf and I are aligned that it's not required they do it, but the PSR has to show and prove that Laravel and Symfony would be supported out of the box, without impact for them.

@mindplay-dk
Copy link
Collaborator Author

It's more that it shouldn't happen (IMO) without (guaranteed) Laravel and Symfony support.

If they won't do the work (which is perfectly fine), you (I mean the person carrying this PSR) would have to do the work. I think @moufmouf and I are aligned that it's not required they do it, but the PSR has to show and prove that Laravel and Symfony would be supported out of the box, without impact for them.

I still don't completely follow, are you saying the PSR must have first-party support?

Or just that it needs to work with those frameworks?

Because, I mean, yes, it should work with all major frameworks - but we're not really in any position to demand first-party support, that's kind of up to them, I think?

But yes, demonstrating that it's going to work with major frameworks, that's probably a must - it's hard to claim this is about interoperability if it doesn't work with major frameworks. 🙂

@moufmouf
Copy link
Contributor

But yes, demonstrating that it's going to work with major frameworks, that's probably a must - it's hard to claim this is about interoperability if it doesn't work with major frameworks. 🙂

Exactly. Which means we need to have an implementation of some kind of adapter for every framework (that can afterwards be added in the core by the maintainers).

The hardest part IMO is to design a PSR that will have a performance on-par with native bundles in the special case of Symfony. They put so much energy in optimizing their container. The PSR should be crafted to not break those optimizations. Otherwise, you risk a rejection from Symfony.

@mindplay-dk
Copy link
Collaborator Author

The hardest part IMO is to design a PSR that will have a performance on-par with native bundles in the special case of Symfony. They put so much energy in optimizing their container. The PSR should be crafted to not break those optimizations. Otherwise, you risk a rejection from Symfony.

@moufmouf I recently had this discussion somewhere (with you?) but I can't find it now.

But once you start putting service providers together, performance (in Symfony or any container) is going to depend on the providers you're using - if all your providers are Symfony's compiled providers, you get Symfony's compiled performance. If some providers were implemented by other provider builders, you get the performance of those providers.

Symfony (to my limited understanding) uses a declarative configuration format - essentially, you're building a data structure from which it then generates code. This type of optimization is necessary because everything is declarative. I could simply handwrite my PSR provider using the same (inline switch statements, as I recall) implementation approach, and my provider would have the same performance. If I can come up with something even faster, I'm free to do that as well.

PSR providers define factories, and (short of crazy kung-fu source code transformations) I don't see how Symfony could possibly apply it's optimizations to anything other than the internal data structures it was designed to operate on?

(I also don't see why it would need to.)

@Ocramius
Copy link
Member

(I also don't see why it would need to.)

The performance difference between declarative and runtime containers is massive: don't underestimate this :-)

A declarative approach also allows programmatic analysis of dependencies, while callback-based DICs are very opaque.

@mindplay-dk
Copy link
Collaborator Author

but so what, you're just making different tradeoffs. that's software. I prefer plain callbacks and readable code, you prefer a DSL and a compiler, both approaches make tradeoffs. almost nothing is universally better or worse than anything else - and micro performance details like these rarely have any measurable impact in real world scenarios.

btw, interop can go both ways - if Symfony wants users to use Symfony-providers only, maybe Symfony's DI container creates providers rather than consuming them, bringing Symfony-level performance to other containers. that still has real benefits - if you want to consume Symfony components, but you prefer writing your own configuration with something familiar to you.

there's no saying everything needs to interop in both directions. (the PSR doesn't try to stipulate that.)

@mindplay-dk
Copy link
Collaborator Author

Thinking more about this, I do think it's unrealistic to expect a compiled container should be able to "compile" PSR service providers - the only way that would be possible, is if the PSR standard was itself declarative, which would be terrible for every container that isn't. It wouldn't make much sense in terms of interop.

But it wouldn't even make sense for Symfony or Laravel, because their declarative formats I'm sure are very different, and we would just end up with something that is overly complex while still being a "lowest common denominator", most likely with limitations for both Laravel and Symfony.

If there's a PSR for service providers, it needs to be simple enough that any service provider can sit comfortably behind it, without getting constrained on either features or performance. I think we've managed this reasonably well for features.

If introducing a performance bottleneck is still in question, I'm curious, did members of the Laravel or Symfony teams point out any specific performance problem with the current (0.4) interfaces?

If there's a specific performance problem, I'd like to understand why.

Someone (I forget who) gave a description of the compiled container as having essentially containing something like:

class UserServiceProvider implements ServiceProviderInterface
{
    public function get(string $id, ContainerInterface $container): mixed
    {
        return match ($id) {
            Cache::class => new FileCache("/tmp/cache"),
            Database::class => new Database(),
            UserRepository::class => new UserRepository(
                $container->get(Database::class),
                $container->get(Cache::class),
            ),
            default => throw new \Exception("unknown ID: {$id}"),
        };
    }
}

If you think about it, this is actually extremely close to ContainerInterface::get, right?

The problem with this approach is it isn't extensible - you could implement a single (compiled) service locator with this approach, but you couldn't combine several of those.

Suppose you were to add dependency enumeration though:

class UserServiceProvider implements ServiceProviderInterface
{
    public function get(string $id, ContainerInterface $container): mixed
    {
        return match ($id) {
            Cache::class => new FileCache("/tmp/cache"),
            Database::class => new Database(),
            UserRepository::class => new UserRepository(
                $container->get(Database::class),
                $container->get(Cache::class),
            ),
            default => throw new \Exception("unknown ID: {$id}"),
        };
    }

    public function getKeys(): array
    {
        return [Cache::class, Database::class, UserRepository::class];
    }
}

Now you can query individual providers and find out which services they provide - which means, if you have a container that collects these providers, it can now decide which provider to call:

class MyContainer implements ContainerInterface
{
    /**
     * @var array<string, ServiceProviderInterface>
     */
    private $providers = [];

    public function addProvider(ServiceProviderInterface $provider)
    {
        $keys = $provider->getKeys();

        foreach ($keys as $key) { 
            $this->providers[$key] = $provider;
        }
    }

    public function get(string $id): mixed
    {
        return $this->provider[$id]->get($id, $this);
    }
}

For that matter, a compiled container could query these providers at compile-time, and this way it could avoid loading a given service provider at all, unless or until it's being used.

You still can't compile the providers into a single provider though - that's still pretty much out of the question except as said by forcing all containers to the declarative and compiled, which, again, offers nothing in terms of interoperability.

But the overhead of unpacking and combining maps of factory functions is removed - you don't need any callables, just one get method for each provider.

Expanding this example to both factories and extensions, the resulting interface looks maybe something like:

interface ServiceProviderInterface
{
    /**
     * @return string[]
     */
    public function getServiceKeys(): array;

    public function createService(string $id, ContainerInterface $container): mixed;

    /**
     * @return string[]
     */
    public function getExtensionKeys(): array;

    public function extendService(string $id, ContainerInterface $container, mixed $previous): mixed;
}

Was something like this already discussed or explored?

Would this mitigate some of the performance concerns?

@mnapoli
Copy link
Member

mnapoli commented Jan 18, 2024

but so what, you're just making different tradeoffs. that's software. I prefer plain callbacks and readable code, you prefer a DSL and a compiler, both approaches make tradeoffs. almost nothing is universally better or worse than anything else - and micro performance details like these rarely have any measurable impact in real world scenarios.

I don't understand your posture.

You're working on a standard. Your opinion on what implementation is better doesn't matter.

The big question here is: what is your goal/what is the goal of the standard.

If you want to solve something for the community, you HAVE to work with the constraints that:

  • providers must work with Symfony and Laravel to gain real adoption
  • Symfony and Laravel each have constraints on their containers (e.g. Symfony = compiled container), the standard must respect those to fulfil the line above (bc that's just the state of the market)

there's no saying everything needs to interop in both directions. (the PSR doesn't try to stipulate that.)

Yeah, there's no rule that a PSR must be adopted by Laravel and Symfony or that it must benefit the larger part of the ecosystem of PHP users 🤷 But I don't see the point.

Also at this point I think I'm giving up on the current state of things here. So many ideas are thrown around, but I don't think there's the right mindset. If one wants to create this standard, one has to do the work. I don't mean to be rude, I just want to be upfront because we're going in circles. I think what's needed here are results, not ideas:

  • does solution X works with which containers?
  • does solution X implies none, little, or many changes to popular containers?
  • what is the performance impact of solution X on the popular containers?
  • would Symfony and Laravel likely agree to solution X and its impact on their code, their performance and their user experience?

@mindplay-dk
Copy link
Collaborator Author

what is the performance impact of solution X on the popular containers?

this is the discussion that was brought up - with my last post, I'm trying to provide a possible answer, how remove any inherent overhead imposed by the standard.

the approach I suggested here removes function calling overhead and reduces the run-time impact on compiled containers to as little as 1 function call and zero callables, which would seem to be the smallest impact it could possibly have.

I'm not personally very interested in (more or less) starting over - I already wrote a full draft of the PSR and meta documents, which would be (more or less) trash if we need to come up with a completely different approach to minimize any (however small) run-time overhead to satisfy the requirements of these compiled containers.

I'm trying my best to consider the problem - despite the fact that this isn't strictly about interoperability.

If one wants to create this standard, one has to do the work.

I think I've been doing the work these past 2 months? and I'm willing to do more, but I need a lead on whether there's any work worth doing - what to explore or pursue next, if anything.

I'm being open to exploring an entirely different approach, if that will please the framework maintainers.

But I wasn't part of whatever discussions you had with them, so I don't know what the exact point(s) of criticism were - I don't even know if the critique was about the current proposal interfaces or a past version. So I don't know if we are going around in circles. Are we? Was this idea already discussed?

I have to ask questions - that's part of "doing the work".

Like, would it be enough to reduce inherent overhead of the standard to essentially 1 function call and get rid of the maps and callables, essentially reducing the run-time overhead (with compiler optimizations) to a single get($id, $container) call?

Or would they demand that an interoperability standard be fully declarative? Because if so, we are dead in the water, as far as I can figure. There is no approach that would work for anyone then. We should hit "archive" and go do something meaningful.

But I don't know, so I have to ask.

I don't know what, if anything, would work, so I have to toss up an idea.

If that's not okay, I really don't know what you expect.

At least I'm trying to do something, okay? 🙂

@moufmouf
Copy link
Contributor

I'm not personally very interested in (more or less) starting over - I already wrote a full draft of the PSR and meta documents, which would be (more or less) trash if we need to come up with a completely different approach to minimize any (however small) run-time overhead to satisfy the requirements of these compiled containers.

Starting over and over again with different approaches is really part of the job (and exhausting). It's only when you envisioned all possible approaches and have been heavily challenged that you can come up with the best solution.
At the very beginning, container-interop/service-provider actually started by a package that was statically describing services (to be stored in YAML/XML files). Look how far we are from our starting point :)

the approach I suggested here removes function calling overhead and reduces the run-time impact on compiled containers to as little as 1 function call and zero callables, which would seem to be the smallest impact it could possibly have.

That's pretty good.

I've been thinking about it a bit more, I think we can do even better :)

Open your mind, this is a change compared to what we have had so far.

The whole point of a container is to lazily invoke services when they are needed.
The risk here is to end up with hundreds of service providers that we need to instantiate, in order to be able to create hundreds/thousands of services.

With our current proposals, when we bootstrap a container, we MUST pass it all instances of the service providers it could use. On each request.
That's because configuration is injected into the service provider constructor.

What if we turned tables around.

class DbServiceProvider {

    #[Factory]
    public static function createDbConnection(LoggerInterface $logger, string $dbHost, string $dbUser, string $dbPassword): DbConnection  {
    	return new DbConnection($dbHost, $dbUser, $dbPassword, $logger);
    }
}

A factory MUST be a static method. I know, it's a shock. But we could view factories as PURE functions.

The important thing is: by analyzing the service provider, one can statically read what the service provider needs (it is the list of parameters passed to functions tagged with #[Factory]).

How are factories/service-providers called by the container? It is entirely up to the container!

One container could implement a function like:

$container->registerServiceProvider(DbServiceProvider::class, [
	"dbHost" => env("DB_HOST"),
	"dbUser" => env("DB_USER"),
	"dbPassword" => env("DB_PASSWORD"),
]);

For another container, it could rely on config files to read values/parameter that are not in the container.

// Values requested by service providers are read from a config file.
$container->addServiceProvider(DbServiceProvider::class);

etc...

In all cases, if a service or a parameter is missing, it is trivial for the container to display a meaningful error message. And doing static analysis of the container setup is easy too.

With this approach, a compiled container would not have to instantiate any service provider, so you get a really good performance: if no services are invoked in the container, the impact is literrally 0.

And some people that really want high performance (Symfony?) could even go as far as turning this back into a declarative format in some specific cases.

Most factories are going to look like this:

    public static function createStuff($param1, $param2): MyStuff  {
    	return new MyStuff($param1, $param2);
    }

If a factory follows that pattern, it could be detected by analyzing the factory AST (with PHPParser?) and turning it into a purely declarative service. It sounds far fetched but I know people who would do that.

Another cool thing: this can easily be extended with additional framework specific attributes.

Symfony has a notion of private / public services that is really specific to their framework. They could come up with a #[\Symfony\Container\Attribute\Private] attribute to make a service private.
Same thing with tagged services (for Laravel)

Another fun fact: if a container does not natively support this PSR yet, a container user could still manually register the factories one by one.

This proposal is not perfect. Compared to current proposals, the main drawback is that the list of services provided by a service provider is hard-coded. For instance, it would be hard to provide a service only if a given PHP extension is installed.

But it has a number of benefits I quite like.
And we can come up with more complex attributes, like:

#[Factory(id: "db_connection")]
public static function createDbConnection(#[ID("dbLogger")] LoggerInterface $logger, #[ID("DB.HOST")] string $dbHost, string $dbUser, string $dbPassword): DbConnection  {
    return new DbConnection($dbHost, $dbUser, $dbPassword, $logger);
}

Anyway, just to say there are plenty of other possibilities and restarting the PSR maybe means comparing those possibilities and asking for feedback on those. People will feel involved if they give you feedback / vote on the ideas and this is a good way to get people involved in the proposal.

@mnapoli
Copy link
Member

mnapoli commented Jan 19, 2024

But I don't know, so I have to ask.

Understood.

It's been years. I don't know, and things probably changed since then.

I think there are two ways to get answers:

  • Build the implementation for Symfony & Laravel, benchmark, and show there is no impact + it's easily implementable for them (i.e. remove possible objections)
  • Ask Symfony & Laravel maintainers (i.e. clear up objections upfront)

Maybe both could actually be good, but anyway I don't have the answers.

@samdark
Copy link

samdark commented Jan 19, 2024

I'm interested and will make sure Yii3 DI Container adopts the proposal. Having you as a lead of the WG would be great.

btw., we have something close to the proposal already:

@mindplay-dk
Copy link
Collaborator Author

@moufmouf the problem with this idea is that, while it minimizes overhead for compiled containers, it drastically increases overhead for all other containers, which now need to not only load all the classes, but then also have to apply reflection to read/instantiate attributes, from every single method.

as I've said previously, I really like the idea! this would be a very nice way to write providers - but it seems more high level than it needs to be for the sake of interoperability? I think it's hard to describe this as a "lowest common denominator"?

not that there's anything wrong with also considering ergonomics, but I think ergonomics should be a secondary goal after low-level interoperability - it doesn't have to look pretty to work.

as much as I like the attribute-driven approach, it is opinionated - I know it is, because it aligns so well with my own opinions, but that's also why I feel obligated to control my excitement, to second guess myself and keep questioning. 😌

for one, this approach offers only one direction of interoperability - all standard providers would need to be written in this format, you can't have different provider builders, and you can't have factories in existing DI containers able to convert themselves into standard providers. I can't write a provider builder that somehow generates attributes, at run-time, that your container can then reflect on - that's not a thing.

so, essentially, this approach gives you one direction of interoperability - my hope is we can offer more versatility, flexibility and freedom to use and combine different approaches. make room for everyone's opinions, so to speak.

case in point, I'm fairly certain the idea you shared here could actually work, in both directions, via the service provider interface I posted above - you could probably even make it cache and compile, like you suggested, and when consuming a standard service provider generated from that, you wouldn't even need to know that.

whenever I see (or think of) an idea in this space, the first thing I ask is whether it could be adapted to a simpler interface - if it can, it's almost definitely not the lowest common denominator, which I think is probably what we're looking for.

a good PSR in this space, I think, is one that has only one opinion, "things should work together", and tries to stay away from pretty much everything else - while making room for everything else, everything from Pimple to the biggest, declarative, compiled, cached containers.

whatever anybody wants to do, I see this PSR as the glue that makes those ideas play together, at a low level.

@mindplay-dk
Copy link
Collaborator Author

I decided to try an experiment based on David's idea and the interface I posted above - but with the added constraint to support caching - here's the resulting (very rough!) POC:

https://github.com/mindplay-dk/foobox

Some interesting learnings emerged from this - namely this crazy type:

array<string,[string,string],array<string,string>>

It's a lot of strings. 😅

If we unpack it a bit, it's something like:

[
    "serviceID" => [
        ["factoryClassName", "factoryMethodName"],
        [
            "factoryMethodParam1" => "serviceIDA",
            "factoryMethodParam2" => "serviceIDB",
        ]
    ]
]

What's interesting is this is all serializable - what makes up the actual bootstrapping in this data-structure is literally just strings, no objects, no callables. (except for that ["factoryClassName", "factoryMethodName"] which happens to be a serializable subset of callable.)

While this POC at the moment has a Container class that combines ServiceProviderInterface instances at run-time (therefore not cacheable) there is actually no reason the merging of this data-structure couldn't be done ahead of time and then be serialized/cached/compiled as well, even "optimizing" away unused services when overridden, even performing the entire validation at compile-time. Many optimization options there.

So I'm not sure the ServiceProviderInterface I suggested above (which is part of this POC) is actually "right", as it appears to get in the way of performing said merge operation ahead of time - what we might need perhaps is something more similar to the internal data structure in this POC?

In a sense, this format is "declarative", and a compiled container could generate literal static function calls directly into a compiled PHP script, if it wanted to - at which point, literally the only overhead is autoloading the factory class and making a static method call.

Note that the attributes on the factory-class is just one possible way to create this data-structure - it could also be done with a declarative API, for example, but I don't know yet how well it'll fit with simpler, callback-based containers such as Pimple. It may be a less natural fit for those? I honestly haven't given it much thought yet though.

There is more to explore in this area, I think.

@mindplay-dk
Copy link
Collaborator Author

mindplay-dk commented Jan 24, 2024

Just to demonstrate that the current proposal can support the exact same factory format proposed by @moufmouf, here's a port of my POC to current ServiceProviderInterface, with no changes to the factory or the test:

https://github.com/mindplay-dk/foobox/compare/current-proposal?diff=split&w=

The down side (compared to my alternative interface idea) is this needs to generate a lot of callables, which causes some (however small) overhead, but also prevents serialization, caching or codegen.

I've been thinking really hard about how to adapt a callable-based container to my alternative interface idea, but I don't see how this could possibly work? If the container API itself uses callables, the provider it generates is invariably going to contain callables, isn't it?

so it's difficult to see how these ideas could ever possibly meld - if it's going to support callable-based approaches to building providers, it's not going to support codegen. But if it's going to support codegen, then it's not going to support callables, which, in terms of using them to build interoperable providers, most likely leaves the majority of containers out in the cold.

if we consider the proposal by @moufmouf more seriously, I would say, this changes the concept from standard providers enabling bidirectional interoperability - only certain container types (declarative) would be able to create providers. Simpler container types (functional, callable-based) would not be able to participate in that way.

on the up side, hand-writing "vanilla" interoperable providers would become much more attractive and feasible - arguably to the point where you might wonder, why would you use a provider builder at all? Personally, I would probably just handwrite all my providers - but I realize that might just be personal bias/preference.

also, this would bring back my original question/reservation about this proposal from 7 years back: would this proposal not basically erase all meaningful differences between containers? beyond providers, all you need is ContainerInterface, for which you could basically have one reference implementation and never need another.

I think it would do that at least for the crop of simple (callable-based) container libraries.

But I can also see how it might do more for more complex (declarative, compiled, cached) containers.

It would be a very substantially different proposal for sure - many pros and cons to think about. 🤔

like for one, this line by @moufmouf keeps lingering in my mind:

Another cool thing: this can easily be extended with additional framework specific attributes.

Can it? I'm not sure it could at all. If you start adding framework-specific attributes, you end up with factories that only work properly with their frameworks. The attributes are metadata, and that metadata format is part of a contract, so you can't just start adding "cool things" if they're framework-specific, can you?

EDIT: also, as mentioned, this approach makes the complex containers faster, and makes simple containers either substantially slower, or in need of more complexity.

I'm unsure how to move on. Is there anything else I could try out or test here?

I'm in experimentation mode now, so I'm very open to your ideas for other experiments to try. 🙂

mindplay-dk added a commit to mindplay-dk/service-provider that referenced this issue Feb 21, 2024
@mindplay-dk
Copy link
Collaborator Author

FYI, I've pushed the refactored interface to a branch for practical testing:

https://github.com/mindplay-dk/service-provider/tree/0.5.0/src

I added exception interfaces similar to those of PSR-11, and (for now) removed the recently added getDependencies as it's not clear to me whether it's actually useful in practice. (and just to focus on the original problem.)

I decided to try out the simplified interface on my experimental container.

The most significant discovery is that, while this approach removes overhead for compiled containers, it might add some overhead for callable-based containers:

https://github.com/mindplay-dk/funbox/blob/d24251e0b8691c1c054e998812d0b0d2f0ca3d79/src/Context.php#L64-L73

    public function addProvider(ServiceProviderInterface $provider): void
    {
        foreach ($provider->getServiceIDs() as $id) {
            $this->factories[$id] = fn (ContainerInterface $container) => $provider->createService($id, $container);
        }

        foreach ($provider->getExtensionIDs() as $id) {
            $this->extensions[$id][] = fn (ContainerInterface $container, mixed $previous) => $provider->extendService($id, $container, $previous);
        }
    }

As you can see, this forces callable-based containers to generate their own callables.

I'm not sure how significant this is? Before this refactoring, providers would generate callables anyhow - so for a callable-based container, this means one extra function-call per service/extension. But I suppose that's true for declarative, compiled containers as well, which would have otherwise baked everything into a single method/call.

I guess this might just be the cost of doing business? Short of actually creating a declarative standard (which would be extremely slow for callable-based containers to consume) I guess maybe this is the tradeoff.

In a sense, performance-wise, the current proposal favorizes callable-based containers, and the refactored interface arguably strikes more of a balance between compiled and callable-based, both taking a very small hit compared to their native implementations.

Difficult to say if this is still worth pursuing though - there has been no real feedback from anyone, the FIG google group appears to be more or less dead, and I don't have the social media reach to drive participation.

Or maybe it's that genuinely no one cares about this subject.

Maybe I'm beating a dead horse.

I don't know, I think I might just give up for now and build what I need as a proprietary framework outside of a standard... 😔

@mindplay-dk
Copy link
Collaborator Author

FYI, I pushed Larry Garfield on this subject a bit. He posted a reply here:

https://groups.google.com/g/php-fig/c/SWIfYEkw89I/m/c9v5EOj9AQAJ

I've posted a reply, which hasn't been approved yet, elaborating a bit on the requirements he outlined - but it sounds like no performance tradeoff would be acceptable for compiled containers, I guess because performance is their main feature, kind of the same thing you guys have been getting at here.

So that leaves only one option, going fully declarative: service providers cannot include any behavior (classes, methods, closures or interfaces) and must essentially be data structures, since nothing else works for the compiled containers - which would mean:

  • Non-compiled containers would be unable to act as service providers.
  • Non-compiled containers would take on extra overhead from parsing the data-structures.
  • Service providers would need to be hand-written, with no IDE support or static inspections. (beyond checking that a class-name or service ID is a string etc.)

I don't think most people would want to write service providers under these circumstances?

But maybe that's okay? Maybe they're only required for a few libraries with complex bootstrapping, and maybe there is still value in making this bootstrapping portable between containers.

I'm not optimistic about it, to be honest, but I am willing to entertain the idea.

Just to get some sense of what we're talking about, I wrote a quick draft:

https://gist.github.com/mindplay-dk/6118034336e32376c62c6ca5f28b9470

It's a far cry from the simplicity of the current proposal - you basically have to replace at least a subset of the programming language with a value-based model, essentially a DSL.

It's not far from what you have to do with the DSLs of existing compiled containers though, most of which also do not provide IDE support or static type-checking.

One glaring issue with this approach is it implies that callback-based containers would be able to somehow reverse callables back to models, which is near-impossible to implement - either that or just accept the limitation that standard service providers can only extend standard service providers.

I don't know if this is worth pursuing.

I'm not convinced something like this is feasible without both sides of the ecosystem making some tradeoffs - if Compiler Camp isn't willing to give an inch on performance, essentially we're talking about a PSR that only works for compiled containers.

I'm not sure I want to spend my time on that.

Would like to hear what you guys think though.

@samdark
Copy link

samdark commented Mar 24, 2024

Usually service providers are used when container configuration logic is complicated and is better described as code. I don't think declarative approach allows that because of being limited in its DSL.

@mindplay-dk
Copy link
Collaborator Author

@samdark yeah, I ended up leaning the same way - which brought me back to the idea @moufmouf aired above, using attributes.

My previous position was that this wasn't necessary, as it can be parsed and represented like the current proposal anyway - but the format might be more compiler-friendly and it's definitely nicer to read and write than anything we've had on the table.

Ken Guest supports it. 💁‍♂️

https://groups.google.com/d/msgid/php-fig/CAKcc2m9%3Dfv44zBQ1ExuaZd6g91VwHfv1U5J%3D6OTYwMvN3DWY2g%40mail.gmail.com?utm_medium=email&utm_source=footer

@mindplay-dk
Copy link
Collaborator Author

mindplay-dk commented Mar 28, 2024

Now that Larry and Ken have expressed some interest in this, could I ask you guys to voice your support in the FIG forum thread please?

For the record, the conversation has turned from callable-based providers, to an AST/DSL-like metaprogramming model, and now back to annotated factory-classes, which might turn out be suitable both for compilers, for callable-based containers, and for human beings. 🙂

@Ocramius I know you don't want to participate in the actual process, but a word of support on behalf of Laminas would mean something. (maybe you could ping mwop and see if he's willing to put in a word as well.)

@samdark likewise, a word from you on behalf of Yii would be helpful.

@mnapoli your support would matter as well, with PHP-DI being probably the most popular DI library with no direct framework affiliation.

@moufmouf given your history with PSR-11, your support might be helpful as well.

I have reached out to Taylor (Laravel) and Fabien (Symfony) but I'm nobody, and they're busy, so they might just ignore me - if you guys would voice your interest, we might be able to at least get a reply from them. It's unlikely they'd be willing to do any work, as they both exited FIG years ago, but third-party/bridged support could still happen, if the community wants it. I reached out to Phil Bennett as well, who maintains League Container.

My hope is we can form a working group and get a PSR number. I don't think there's any hope of drumming up interest from the PHP community unless there's an official working group and an organized effort to make this happen, so that's what I'm pushing for now.

Cheers :-)

@samdark
Copy link

samdark commented Apr 2, 2024

@nicolas-grekas maybe you'd like to participate or leave your opinion on behalf of Symfony?

@GrahamCampbell, @driesvints same question about Laravel.

@driesvints
Copy link

Hey all. I really appreciate the mention and offer but atm I don't have the bandwidth to take this on.

@mindplay-dk
Copy link
Collaborator Author

@driesvints we're not asking you to join the working group (though you would of course be more than welcome to!) but at this time just need some indication from framework representatives (on the FIG thread) in order to form a working group in the first place.

The major framework representatives were historically against this proposal, mainly on the grounds of performance limitations, and we're coming up with new ideas that might could address that - but the idea needs a working group, it needs community participation and support, which it will not receive unless a working group is formed for community members to join and support.

Core FIG members have indicate they would be interested in forming a working group, but not unless there is some indication of interest from the major framework reps. You don't have to do any work, we just need a word of support for the idea. 😄

@nicolas-grekas
Copy link

nicolas-grekas commented Apr 3, 2024

Same as a few others here: I'm only interested in talking about code. At the moment, I don't get the problem this is trying to solve for the community so I can't say Symfony would be interested (I'd have a hard time explaining why this is needed.)

I'm not interested in trying to formalize how definitions should look like. That's not something that needs to be standardized to me.

There's one community goal I do understand, and I'm not sure this is what is being talked about here:

There are some libraries that are complex to use (e.g. Doctrine) and that would benefit from having a DIC. Doctrine is the canonical example: it uses the new $class configuration pattern a lot because it has no DIC and this makes it complex to configure at will. Proper DI (passing instances, not $class names) would fix that, but the reason why Doctrine works this way is because it wants to provide some "easy" way to bootstrap the thing without a full framework. Then, frameworks can integrate Doctrine by maintaining an integration bridge (DoctrineBundle for Symfony), which is another non-trivial piece of code that does the mapping between Doctrine extensibility points and the framework's DIC (and its configuration system).

It might be beneficial to the community if there were a way to write this glue once in Doctrine and make it pluggable in compatible frameworks.

In symfony/contracts, we do have a ServiceProviderInterface built for this purpose:
https://github.com/symfony/service-contracts/blob/cea2eccfcd27ac3deb252bd67f78b9b8ffc4da84/ServiceProviderInterface.php#L33-L44

This interface is just about exposing what a container knows about in the form of service-id => type mappings. This lists the identifiers you can $container->get() basically, and what type of value you'll get in return.

Then we also have a ServiceSubscriberInterface which allows a service to tell which identifiers it will "get()" from the container it has as a dependency. (Note that we also recently added an AutowireLocator attribute to describe the same thing, it's a nice companion to ContainerInterface.)

Back to my Doctrine example: in some hypothetical future where this is more standardized, Doctrine could ship a class that'd implement both these ServiceProviderInterface and ServiceSubscriberInterface. This class would be a container that would be configurable thanks to the subscriber part, about would be wireable into any other app thanks to the provider part.

With the autowiring mechanisms we have in both Laravel and Symfony, Doctrine would then just name this class in its composer.json file to make everything work in a full automated way: install Doctrine and consume it.

Some last words: all this is a mental construction that might very likely be not worth the effort ;)

@taylorotwell
Copy link

Hey all! 👋 I basically agree with @nicolas-grekas that I'm not sure this is worth the effort.

My hunch is that packages that have such complex bootstrap requirements are probably pretty rare, and most packages can get by with a couple of small framework integration hooks / classes like they do now.

My personal two cents is that PHP FIG has generally served its purpose, the reward of which was PSR-0, 1, 2, 4, and 7. I think we are probably at the point of significant diminishing returns on further PSRs.

@Crell
Copy link

Crell commented Apr 4, 2024

@nicolas-grekas The Doctrine example you give is a good one, but I disagree that it's an edge case. Any multi-class library would need some kind of framework specific bridge. That is indeed exactly what a common standard would eliminate.

For instance, my Serde library, if I wanted to wire it up "cleanly" in Symfony, would require registering at least 3 services from 2 libraries, one of which is an interface so I'd also need to register that and map it to the right class by default. And someone using it in Laravel would have to do something different to do the same registration.

Multiply that by all the libraries out there that have "small but non-trivial" configuration, and it adds up fast.

If instead of bridges or "copy this code into your YAML file", Serde could just document "Call $yourContainer->provider(new SerdeProvider())" (or something), and that works the same for Symfony, Laravel, Laminas, Yii, and Slim, then it becomes a lot easier to produce "framework agnostic" libraries that can easily slot into any framework, increasing the number of libraries available for a framework and the number of frameworks supported by each library.

That's where the benefit is.

Most of the debate is about how that would look internally, and the tension between the robustness of definition-based registration (a la Symfony) and closure-based registration (a la Laravel). The former takes more work, but is a requirement for "alter hooks" (compiler passes) and compilation, while the latter is easier for simple cases but notably slower and less flexible.

One possible option is for a PSR to provide both options via separate interfaces, and explicitly allow frameworks to support only one or the other as appropriate, but they could do both (as definition-based can be converted to closure very easily, as I demonstrated on the FIG list). It's not ideal, but it may be the most productive way forward.

@jaapio
Copy link

jaapio commented Apr 4, 2024

I do fully agree with @Crell that it would be very usefull for many projects to have an easier way to integrate with the frameworks. If you just look at packagist how many different symony bundles and zf modules and Laravel wrappers are published and maintained by many different users it would be very useful to have a more general solution for this problem. But I'm not fully convinced this PSR will be able to rule them all.

I read a lot about service providers and ways to do this. However in the PSR is also something explained about extensions. And this is where things might become more tricky. I never wrote a DI container myself, but I have a lot of experience with the DI component of symfony and used the zend framework service locator for quite some time. And the main difference to me is the whole approach. While the Symfony container can do a lot of "magic", The service locator of ZF and currently laminas is way more straight forward. They are actually explained as two different patterns in dependency handling. (DI vs Service Locators). I think it makes sense to try to distinguish between which pattern you try to cover with this PSR as the approach might be different. From a ContainerInterface point of view there is no difference between a DI container or a service locator. However behind the scenes a DI container is mostly more complicated as it tries to do the autowiring in general?

Another concern from my side is that many frameworks and DI-like solutions have a concept of tags or other ways to collect services that implement an equal interface to inject them as an array/generator/iteratable in a service that will consume them. This might be a whole different topic to cover, however I would like to mention it as it might be an conflicting approach to "just" factories.

Maybe we should first focus on a way to provide a generic FactoryInterface together with a ServiceProviderInterface that allows uses to register services in a compatible container. Maybe I didn't read all what has been written before good enough or I didn't get the message somewhere. If so, please let me know.

Apart from all of this, @mindplay-dk thank you for all the effort to get this starting again.

@Crell
Copy link

Crell commented Apr 5, 2024

DI vs SL isn't really relevant here. That distinction is entirely on how one uses a PSR-11 container; it's all about getting stuff out, not putting it in. I don't think Symfony has any appreciable "magic" in this regard, compared to others.

Autowiring is also not really relevant here; these days any good DI container should have it, but that's only tangentially related here. (I suppose it becomes relevant if we want to say "the definition can leave stuff out and expect it to be autowired". But that's the kind of question a WG would need to sort out, not something for us here.)

Extensions/compiler passes/alter hooks are an interesting duck. Only some container support them currently. If we were to include that, it would, realistically, mandate using definition-based registration as there's no useful way to "alter" a closure.

Tags et al are an interesting problem space; I'm not sure what a PSR should do there. But again, that's not a now-question. That's a question for a WG to deal with.

I do feel the need to remind everyone that a PSR that says "do X" does not in any way shape or form preclude an implementation that supports "do X or Y". Symfony and Laravel can absolutely continue to provide their own current registration mechanisms in addition to whatever PSR gets developed. There's no inherent conflict there.

@nicolas-grekas @taylorotwell Does any of this help clarify the goal for a PSR WG?

@mindplay-dk
Copy link
Collaborator Author

I do feel the need to remind everyone that a PSR that says "do X" does not in any way shape or form preclude an implementation that supports "do X or Y". Symfony and Laravel can absolutely continue to provide their own current registration mechanisms in addition to whatever PSR gets developed. There's no inherent conflict there.

100% 👍

since early discussions about this proposal, it's been a clear non-goal for this standard to replace anything else - the intention was always mainly for libraries to be able to ship their bootstrapping in a universal format, rather than having to ship bundles or integration packages for every framework and DI container out there.

PSR implementations are a good example - for example, HTTP factories could ship with one universal provider, and some PSR compatible logging frameworks could ship with (potentially many) providers, all of which would plug in anywhere.

With all the implementations of various PSR standards, being able to ship these with ready providers and bootstrap them correctly anywhere with 1 line of code seems to me like a natural continuation of the PSR interoperability philosophy.

again, this would not exist to replace DI containers, and I don't imagine most developers would want to write universal providers for their own application code - for that, most people would want the container that goes with their framework of choice, or the container you've chosen to use, I think.

I do agree with Taylor that there isn't likely going to be boatloads of packages that need this or benefit from shipping a standard provider - but I do see a potential real benefit for library developers, being able to ship libraries that work anywhere, out of the box, freeing developers from having to learn and manually integrate every framework and DI container out there.

image

@mindplay-dk
Copy link
Collaborator Author

I just published a draft and a working implementation demonstrating the attribute-based approach:

https://groups.google.com/g/php-fig/c/SWIfYEkw89I/m/_6XPs6Y5BwAJ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests