Lithium Package and Dependency Management

[daschl]

What about using http://packagist.org and composer to manage plugin dependencies? It looks like it is getting more traction and maybe we can integrate it with the Library class too. Also, if people are encouraged to add their plugins there too we have a "central" repository of libraries which can easily be reused.

This is marked as an RFC and it would be cool to get comments both from the core devs and the community to see what they think about it and how they want to use it.

Regards,
Michael

[mehlah]

Had same thoughts few days ago...
What will become li3_lab ?

[greut]

Another one, work in progess: phark

[marcghorayeb]

packagist seems quite nice :)

[daschl]

This doesn't mean that li3_lab is not needed. I packagist is just a repo for composer, so we could also do a composer repo which can be accessed through the lab. Just thinking :)

Because even if we roll our own stuff, we should think about how we can easily integrate other packages (aura components, symfony,..) easily.

[mehlah]

Fair enough :-)
👍

[mariuswilms]

Before I begin let me say that I think package managing is complex. That's why I think it's a good idea to leave this to external projects - so we've got more time to focus on the core and nearby plugins. We shouldn't underestimate how much resources maintaining a separate package manager cost. Also I think the composer devs are doing it right in that they make it a cross framework/PHP project.

1. We should extract any packaging or lab specific code from core into li3_lab. I'm looking at:
   https://github.com/UnionOfRAD/lithium/blob/master/console/command/Library.php#L300-529
2. We should move the extensions feature from li3_lab into lithium_bin (aka pastium.org) so that lab is solely managing entire packages. lithium_bin will need a few tweaks i.e. it must gain the ability to add further meta data like description, maintainers (plural meaning multiple - with fields name, email and website). It further must be possible to mark a paste as an extension. This should be easy to accomplish but two questions come to my mind: Is lithium_bin not the place for transient pastes? Do we actually need extensions? In general I think it's good if lithium_bin gets more attention it also seems a pretty hackable project, people like to work on.
3. On the server side we'll reimplement packagist on lab.lithify.me. That means we keep the idea and look of the lab but reuse as much from the packagist code as possible. This could happen by actually pulling in packagist as a library and using/wrapping there classes or writing our own packagist implementation. I didn't look through the packagist code, so someone with knowledge could post her/his estimations about that here. As an alternative I'd also consider using packagist.org but in that case lithium plugins may not get the attention they deserve.
4. On the client side we'll make the lab command in li3_lab wrap around composer or we decide not to provide any client at all and use composer directly.

Bottom line: We should only deal with packaging as much as needed and make our packaging layer as thin as possible while making the bin much more stronger and moving code to the place they seem to fit best. To "start from scratch" makes no sense to me here but changing the inner workings of the lab does - so maintaining gets easier.

As a sidenote: I actually worked on 1. and 2. In the case of 1. my efforts we're related to how packaging is related to a CI server I was writing. In the case of 2. I got only so far to prepare and update the bin but not actually execute migrating the extension feature. This would've required more discussion which I missed starting until now ;)

Historical sidenote: The lab introduced an equal format as composer does now with their JSON configuration. I still think the lab is great but we just hadn't enough time to improve and battle test it further. I still like the notion of laboratory and thus the title of the lab.

[daschl]

To get a feel for composer and packaging with Lithium, I wrote a short blog post.

http://nitschinger.at/Playing-with-Composer-and-Lithium

After playing with it, I really like how it works and what it does.

[mariano]

I totally agree with @daschl

Composer is getting a lot of heat, and for good reason. Adding li3 to packagist would be recommended

[d1rk]

Sounds like a good idea. One of the current problems of php is packaging. Solving it on a framework level (which is done with a good integration) makes it a good fit. I like davids ideas and thoughts so far. That would make a start easy and realistic.

From a developers point of view, i would like to add my personal needs:

1. get packages as easy as possible, like described in daschls post
2. package new packets as easy as possible, like li3_lab allows
3. package a project, so it can be used as a package, via packagist.org or as a .phar-file
4. maintain future versions easily and get information about new releases for other packages

That said, i have not dived into php packaging that much, especially because there is no easy way to do so.

[josegonzalez]

For the record, I did a bit of investigation of various specs - although this is by no means exhaustive - and I hope to convince the composer peeps to make some changes to their "spec" so that some things are a bit more flexible than they currently are.

Results are at https://github.com/josegonzalez/dependency-management . Feel free to have a laugh.

As far as keeping the li3_lab around, but having it a host for li3-specific stuff. I also spoke with the composer peeps on the matter. There are potential naming collisions, as people use different names online and in different communities, we might all use the same username. I've proposed some changes to add a community to the name key, but otherwise people will be fighting for their name on the official packagist site, or we'll have to base it on some outside user management tool.

I think the best bet would be to have li3_lab be a packagist alternative for lithium-specific packages. This would be good so that lithium can start guiding composer's development towards being a great PHP solution to dependency management, and not just a good Pear dependency management alternative.

[nateabele]

@josegonzalez What's the status on your "community" amendment to the Composer spec? Did they end up integrating it? If not, is it something we could start moving forward without? For us I don't think it'll be too huge an issue, since most Lithium plugins and such are prefixed.

[josegonzalez]

http://getcomposer.org/doc/04-schema

See the "type" key. We'd want a specific type for lithium. Or two. I think we have libraries and extensions, no? Maybe something like li3-library and li3-extension?

[nateabele]

Extensions are individual classes which typically are (or should be, anyway) distributed in a library, so we can get by on one type value.

[josegonzalez]

li3? The type we use is up to us.

[nateabele]

I think lithium might be better, as it's more explicit, but yeah, that's fine. So what are our next steps? Anything I can help with?