Beginners being herded into command line due to composer #4950
Comments
|
That's not 100% true. A lot of end-user applications are packaged with their dependencies and anyone can download and run them without running composer.
But the development of all these great products got benefits of using composer to handle dependencies. |
|
I am not sure about the minority/majority side of your argument. That's a very biased viewpoint. From where I stand the majority are fine using the command line. Anyway.. there are tools to interact with composer without command line. PhpStorm and possibly other IDEs have plugins that let you do everything you need to do from the editor for example. http://desktop.sensiolabs.org/ used to do this too but it seems they have abandoned the project, maybe because it wasn't useful and used enough? My point is that there is nothing preventing someone to building such a tool. We work on composer as a low level command line utility, and if people feel like building GUIs (some CMSs have I believe) for it then that's great. Blaming us for not doing it though is not fair and pointless. There is only so much time we can invest in these things. As for forcing people into using composer.. I sure didn't do it. Application developers are switching to it because they see it as a convenience, so if your application is now only installable this way I would suggest you take that rant over to them. I can't force nor stop anyone from using the tool. |
|
I'm not against developers including composer install as an extra for those inclined to use it. But that is not how it is being used anymore. It's now being used as a barrier to entry. Besides offering convenience to a minority, it offers nothing. Beginners aren't looking to jump over command line hurdles to achieve a minor convenience. They are looking for a foot in the door - a "Hello World".... And blaming the application developers is pointless, while you keep facilitating it. I see nowhere any message to developers that composer should only be used as an extra way, rather than the only way, to install something. |
|
It may take some time to understand Composer/CLI when you first encounter it, but the reason everyone is moving towards Composer instead of bundling dependancies, that it's just a lot easier to update. The old way:
The new way:
So imho it's a lot easier. And the more libraries you use, the more problems you get with the 'old' method. You get conflicts without even realising it, packages that include the same dependancies (but different versions) and you manually need to check for updates. If you don't want to learn how to use the command line, don't become a developer. But seriously, it's just a few commands and 1 line of code to register the autoloader.. |
But why is that a barrier? Why is a GUI application so much easier to learn than essentially 3 command line tools? What's easier about selecting A beginner can do everything they'd ever want to do realistically by just using
Right click, create new file, name it
That's because it's pretty much unreplaceable. It fills a gap that simply wasn't filled before. Before Composer you had to download 25 libraries, try to get them to work together, to load together, and basically stumble in the dark for hours to hack it into submission and pray that it would remain stable. Since Composer we type How can that not be an improvement? If anything, Composer has made PHP development far more accessible to starting developers than ever before. |
|
If the CLI is a barrier for entry, then either your motivation is lacking, or development is not a career for you. Even javascript, most commonly associated with frontend development, comes with lots of CLI tooling now a days. Expand your horizon, or find something else that does genuinely interest you :-) |
|
Wasn't this issue about application developers requiring Composer knowledge? Thus, we're talking about users of PHP software needing to get to grips with Composer to install the software. That is, in fact, a problem and something we're dealing with at Flarum, too. (Lots of negative feedback from users who flock to PHP forum software because it's dead simple to install. Composer does not help here.) |
|
Nothing is stopping you from providing builds with vendor dependancies included, right? Sames as the examples given, like Drupal, Silex etc. |
|
Another massive benefit to using the command line is that in the main, what I type is identical for Windows, OSX, Ubuntu, etc. Building GUI applications for all these different platforms presents a whole set of other issues. Chief of which is getting functional parity across all platforms, keeping up to date with all the security issues that those OSes get from time to time. On your headless server, accessible only via SSH, how else are you going to install applications? The command line is just another tool, which often provides a far greater degree of flexibility. Ruby has gem, Javascript has bower (probably others, I'm not sure), Perl has CPAN (again ... I think ). So all of these languages have a CLI to deal with package management. RPM, YUM, homebrew, etc. more of the same. All of them are just tools. That are easy to install, configure, use, upgrade, etc. Without the CLI, I can't imagine I'd have as much control in installing applications. |
|
@barryvdh That's true, but that doesn't help much since we're also using Composer to install extensions. Thus, we need Composer bundled with the project etc. Back on topic: |
|
@franzliedke I definitely see that point, and I don't think I misunderstood, but I also don't see what we as the Composer project can do to help fix this. I also don't really see it as our problem. Application developers could team up and work out a good solution, as you know the problem space way better and you have the incentive to fix it. That's how open source works. |
|
I can fully understand the perspective of using composer if you're lazy developer. But let me add a complaint about this massive inclusion of composer in mayor PHP projects seen from the perspective of a server admin: If you maintain a couple of servers (in my case its just 10+) and you have a system with an existing package management that included quite a huge list of PHP software for years and is dropping support of more and more PHP software due to the recent spike of composer usage, this is frustrating! Composer might be a good tool for development but in my opinion its a very bad choice for hosting. Maybe its my lack of experience with composer but i think there is a huge gap of proper dependency handling. Its pretty annoying if you have to run custom package managers for every software that comes around since you want to be able to maintain your servers with simple steps. Can you image how painfully it is to maintain servers this way? Open source software should be shipped in plain source without its hard linked dependencies that will break when using those sources without a forced package manager. https://wiki.gentoo.org/wiki/Project:PHP/The_Composer_problem |
|
@sint that's another issue entirely IMO, unless sysadmins are getting the CLI-scare as well these days, but I'll bite. For packaging I think you should talk to @remicollet as he did quite some work in that domain for RedHat and I'd bet a lot of it can be re-used for other distros. Again though this is not something composer can or should fix IMO. We've worked with Remi when needed to facilitate the process, but we are not the ones that can or should lead this effort. |
|
@Seldaek: well, it might not be exactly the same problem but the idea is the same. Composer should encourage people not to use composer for shipment! This is wrong! |
This statement is deprecated if you ask me. The days of "global" dependencies (e.g. kind of how PEAR approached it) are over. Projects specify their own exact requirements now, and should be bundled and deployed as one package. |
|
Lets not drift off-topic though :-) |
|
@alcohol: But you still have servers with linux and you still have maintainers other then the developers to maintain your software in their linux distribution. It was this way and it will be this way for quite a while. So its ok if you ship included libraries but please don't force users to download you software with composer only and break any other way of setup the way composer does it. That way it would still be up to the maintainer of each linux distribution. |
|
Sorry for beeing slightly offtopic. I still think its the same problem: Forced usage of Composer as OP has stated but with different intentions. |
|
Well to be completely honest, I only see one problem with https://wiki.gentoo.org/wiki/Project:PHP/The_Composer_problem -- and that is people. Clinging to their old ways. |
|
GMTA @alcohol - I was about to say about that Wiki that it was written by the same people that protested when telegraph was being replaced by email.
Progress happens, yet people are conservative - those 2 things collided in the Roman Empire and they will keep colliding when we have colonized Mars. For PHP developers Composer is finally a good solution to a years old boiling problem. It may not be perfect, it has room for improvement, but it's most certainly A Good Idea(tm) to use and deploy it today. |
|
I'd like to enlight that composer and downstream distribution IS NOT a problem (and I disagree with the gentoo problem description, which is mostly against "autoloader") The composer.json file gives very useful information about dependencies. For exemple, I recently encourage phpMyAdmin team to switch to composer for its external libraries. Of course, they will continue to distribute a "full" archive, with all bundled libraries, as always. But it will be easier for downstream to know (looking at composer.lock, which versions are required) |
|
It seem you don't get the point. You guys probably never managed any server other then some half hacked ubuntu. In my opinion there are 2 types of server admins, one that focuses on deploy, deploy, deploy (like those docker guys) and one that just has a couple of stable servers running with long term projects.
@alcohol: please don't tell me about "old ways". In 10 years of hosting i've seen so many half backed hipster ideas coming along that where gone just a few month / years later. What happens if composer is not hip anymore? I did not intent to move this into a hosting discussion as it was focused about forced shipment and i still think it would be nice to have PHP software shipped without composer and i don't see any problem with this. |
|
Everyone takes 5min and calms down a notch please, I don't want to hear about how the other person's views are irrelevant or they never managed enough servers to have an opinion. You are all welcome to discuss in a civilized manner. @sint please see feedback by @remicollet and talk to him directly for more info if you are interested in handling this better in Gentoo. I am not going to question the validity of system package managers, that's none of my business tbh, but I don't think Composer goes against them, it's just a use case that has to be made to work. |
|
I've managed plenty of Linux systems, with distributions ranging from Debian/Ubuntu to CentOS and Archlinux, for quite some years already. So yes, I do like to think that I have some experience in the area of "managing servers". At one of my previous employers we also used Composer in our development flow. We also had our own SPEC files to wrap all in-house made PHP projects in neatly wrapped rpm packages. Due to the nature of PHP it is simply not feasible to have shared global libraries, so it makes perfect sense to package everything together on a per application/project basis. |
|
It would seem that application developers need to be able to bundle all their dependencies into their application installer. How is this NOT possible with composer doing all the grunt dependency resolution and the application developer keeping their composer.json file uptodate and then using whatever installing tech they want, I'm not sure, but I'm not an application developer. One thing that I think WOULD/COULD/SHOULD be handled by composer is the ability to export production content - so no tests, documentation, etc. Just the code and nothing else. (Similarly to the functionality of bower-installer for javascript). This would require each package's composer.json to be correctly setup, advertising the different elements (src, test, docs, etc.). And maybe a bridge between packing-installer-x and composer to allow for migration/support of composer within the older style application package manager. As a developer, I can't see myself moving from composer any time soon. I dropped PEAR usage a LONG time ago and there really isn't a sensible/viable alternative. So, if we do accept that there is something missing between composer and the application maintainers, what is it that you need? Let's see if that bridge can be built. |
IMHO, this can also be achieved through well written SPEC/Ebuild/etc files. Moving this into Composer gives developers more control admittedly, but that is something that can be argued both in favour of and against. |
|
When libraries set up their .gitattributes with export-ignore, the tests/examples/docs are excluded from the dist archives.. |
|
@alcohol: i have seen feedback by @remicollet and i can understand his point. But he is not using Gentoo so he most likely don't care. Even if the Gentoo way might look "old" or "edge case" for you, they had a working package management system over 10 years ago with focus on fine grade dependency management. You might not like those suggested solutions over there but the problem still is unique to PHP software bundled with composer. Gentoo works quite well with python software even if they use PIP. As said before i haven't used Composer for development. Is there any way to create a final build ready for shipment with Composer? Would it be much of a difference to ship those final builds with those missing require statements or a proper autoloader? |
|
While no answer was given to my last question i was checking the Composer documentation for a way to create final builds of software written with Composer included in development. It seems there is no way to render a final build ready for shipment but in your own documentation you are stating the following:
I have no problem with that but it makes my point even more clear that composer is not made as a replacement for Apt or Yum and should not be used to install software the way Apt or Yum (or Portage) is doing it.
I would suggest some additional command within Composer to create final builds. This with Linux Maintainers in mind. People on Linux still use their given tools if possible. You would not only encourage people to create final builds of PHP software but create a default way of doing so. These builds should include all dependencies statically linked and a file like composer.json with dependencies and its versions listed. This should give any external Maintainer everything they need. I am not quite sure about the linking of dependencies. The autoloading seems to make problems and those suggestions in the Gentoo wiki article might be arguable. But it would be much of help you would keep in mind that Gentoo is around for some 10+ years and even if you don't see any point in having dependencies managed outside of composer, its their concept of running Linux. Additionally it might be the case that there are more package managers and maintainers out there in Linux that handle dependencies externally. I don't see a problem with shipping dependencies in the final package but it should be possible to link them differently if needed. |
That is just ignorance. Or opinion. In truth, there is no one true way. I am strongly opinionated as well. Composer has no place on production environments as far as I am concerned. We do not document packaging procedures because we believe those are best left in the hands of the maintainers. There is not just one way to distribution, and hence not one way to packaging. This is exactly why Composer considers itself a dependency manager only, and not a package manager. |
|
@alcohol: but without providing a way to deploy to production, people will just install Composer on production servers (and force others to do so as well) just because they don't know otherwise. |
|
I see a huge difference between custom projects and software delivered to users. If you work on a custom project you most likely use git for deployment. This keeps composer out of production servers. But for delivered software there is no general solution given for PHP rather than taken away the ones that existed. |
|
I've just checked what Arch Linux is doing. And it seems like you'll end up with Composer on every system as well if you want to install drush. This leaves composer on production server just because people are left with no other option. I know Drush is providing a Phar now but its an example about the general idea. See here in their listed dependencies: https://aur.archlinux.org/packages/drush/ |
|
As @alcohol said he is highly opinionated on the issue of whether Composer belongs on production servers. I for one agree with that statement in an ideal world, but acknowledge that deployment workflows without Composer are not realistic for environments with many smaller sites. Therefore it makes perfect sense imo to use Composer on servers as out of the box behavior - if you want custom stuff go custom, nobody's stopping anyone from bypassing package management if it doesn't suffice for what you want to achieve. |
|
The zip files projects used to offer already contained external dependencies, and further back PHP projects hardly had any because using dependencies was so hard, so they just straight up copied the code into them. Composer formalises this and makes it easier to update them. So even way back when, the distribution package managers were hardly used to handle these dependencies, they were mostly invisible to distro maintainers anyway. So that hasn't changed at all, but now distro package maintainers realize that those PHP packages they were maintaining so long, actually use a ton of dependencies, that they never knew about before. When they didn't know about them cause they were just commited in the source code, they didn't care. So I suggest we just stick to that practice. A phar file is the same thing as the zip file they used to publish. You can't in their current state let the distro package manager handle dependencies on libraries that you need in multiple major versions for different PHP applications. So contact the respective projects, let them generate a zip or a phar file for you that contains dependencies and the autoloader, and you have what you had before. Or simply run that one composer command to give you the finished zip file. |
|
@naderman: thats simply not true. At least Gentoo maintainers removed obvious included dependencies and set them as a Portage dependency for that package. They've handled with every additional problem themself. Might it be security problems within dependencies or something else. This might look strange to you but it makes perfectly sense if you look at the concept of Gentoo. It is similar to your concept and even if Gentoo is not as popular as Ubuntu it was for example used by Google to create Chrome OS. Whatever, for some reason it is just as frustrating talking to you guys as it is frustrating to see more and more packages loosing support within Gentoo. None of you have any experience with Portage but yet you claim its some obsolete old way and Gentoo should change. You preset some not tested magic find commands that would do the same things that Portage does for years and in the same sentence you advise against it. We are talking about PHP here. A language with a long history of security problems made to face to Web. But yet most of you have this shouldn't be problem attitude when it comes to dependency handling within security problems. Sorry if Gentoo is not trusting PHP developers fixing this ASAP. Gentoo has its focus on stability and security and as an administrator i am the one to blame if the server is hacked. No one cares if this was caused by improper handling of a developer on vacation. Also no one cares if Composer is causing the lack of proper PHP software available on my servers. There has not been any problem with any PHP package i've been using with Gentoo over several years up until Composer arrived. Isn't that strange? Please spare me your telegraph examples again because even if telegraph is superior to written letters, message where written down to letters in the end because most people do not talk morse code. I would have loved to see some progress regarding this situation but it seems to be a lost cause. |
Some, maybe, but I know of a number of dependencies that were bundled with phpBB (the package I have most first hand experience with distro maintainers with) for a decade, that were treated in exactly the way I described. Now they will be Composer dependencies, and we provide a built zip file that still includes them just as before, and uses composer to install them into our zip file. So like I said no real change.
I never said anything about Gentoo in particular, this is a subject for all distro package managers.
Not sure why you think I don't have any experience with portage. My point would be, use the right tool for the job. Rather than try and fit everything into Portage, when other tools can handle some things better. Hence, use the zip or phar file built with composer if you want to install a PHP application with Portage. Let PHP projects handle their dependencies internally.
Starting to wonder why I even bother replying. I mean apart from distros patching upstream packages to contain security issues and bugs that weren't even in the upstream package, you can rebuild your zip or phar with updated dependencies if you want to, even when using composer, you just have to accept that that's the tool used as part of building PHP applications.
That's not strange, you can keep using PHP packages the same way you always have, just use composer to install the dependencies, because they're not inside the source code repository anymore, and then package the result, no change.
Right ... |
Maybe you should consider to stop calling Composer 'inferior' and provide a constructive solution to a concrete problem. The people here are trying to help you and and provide suggestions, but I don't think this is currently going anywhere.. Perhaps someone should lock this issue, as every argument has been repeated for 12 times right now.. |
|
To reiterate two points, to no one in particular: First, PHP is not special. The language is not special, the community is not special, and that has not changed. We have tens of thousands of packages, many of which use PHP, that already work with global dependencies. It's still easy to package any project that doesn't use Composer, or provides an autoloader to replace all of the Second, bundling dependencies in a big zip file does not fix the problem. There's no point in using a package manager to ship a zip full of bundled dependencies; the point of the package manager is to manage those dependencies. You may have to take it on faith that this is useful to people with a large number of installations. The alternative is basically to keep a post-it note with a list of the thousand places where I left phpunit.phar and then check them every morning to make sure that none of them contain critical bugs. That becomes a full-time job if you need to guarantee the security of your systems. @remicollet Thanks for the pointer. We have one critical package, phpunit, where someone accidentally committed the phar archive (not realizing it bundled its dependencies) and now everyone depends on it. I realized yesterday that we may be able to use Debian -- or now Fedora, I guess -- as our upstream to offer a version that doesn't bundle its dependencies. Any other major packages that Debian or Fedora provide, we could pilfer in a similar manner. I respect the amount of work that you guys put into that (better men than I), but manually writing autoloaders for every package and every dependency falls foul of my "If your solution involves hours of manual work per package, forget it, I have better things to do." |
|
I think this discussion going in circles...
This is all about upstream / downstream collaboration. Good project are aware of this and are usually fine/happy with this collaboration. Now I really don't understand what is the Gentoo issue. Packaging libraries and applications is possible, and IMHO, composer helps a lot with a nice description of the dependencies. |
|
@orlitzky Think of Composer dependencies as source dependencies, which are compiled (well, archived really) into a release binary (well, zip file really). Shouldn't your package manager only take care of the other binary dependencies the package needs to run? I'm thinking PHP, webserver, whatever... Composer as a dependency manager is then responsible for resolving the dependencies the maintainer needs to create a functional application, before the build/compile step. |
|
@franzliedke That would be analogous to a statically-linked binary, and those have most of the same problems as the bundled dependencies that we're trying to avoid. For a current example, take CVE-2015-7547. That's a very bad bug in glibc, but on Linux systems it's easy to fix, because you just upgrade glibc and the fix is automatically applied to the thousands of packages that use glibc. That works thanks to dynamic linking; those packages all load the one (globally-installed) system copy of glibc at runtime. By fixing that one copy, you fix everything that uses it. Static linking would have you essentially copy/paste the code for glibc into the binaries of the thousands of projects that use it. If all of the programs on your system were statically linked to glibc, there would be no easy way to fix that bug. You would have to recompile every package that uses glibc from scratch, assuming you had some way to do that. That's how Composer works, except there's no linking involved. Now imagine a world where the ~1,500 packages on every system were all installed with Composer, and they all had their own copy of glibc. (Pretend that Composer can install glibc, the details are irrelevant). How would you patch that? On 100 servers? What I'm getting at is that no amount of mental gymnastics will help in this case. A true package manager is your only hope, as long as you let it actually manage your packages. But by bundling twenty other dependencies along with each PHP package, you prevent the package manager from doing its job, and therefore doom yourself in the event of a critical bug like that one in glibc. It doesn't matter how you think of it, you're still fucked =) |
|
Gotcha. How many servers run that many web applications that are managed with Composer, though? |
@orlitzky I think distribution collaboration is also a nice things. More eyes, more hands, more QA. |
|
Would something like this help ...
A specific version of a dependency is installed once and all applications access the same files for that particular version. One thing I don't get though is that, taking the glibc example, you can't make any significant changes to the shared library and expect all users to automatically support it. Only limited bug fixes. But things can still break. And multiple applications would break. |
|
Some great stories here (that's all I can make of them to be honest, without any factual information). @orlitzky your CVE example is amusing, but not really applicable to the context of this discussion. Software evolves much more rapidly now a days, and as such, new versions are also released much more frequently. It simply is not reasonable to expect that those 1500 projects running on your server are all using the same version of any libraries they have in common. And since globally installing multiple versions is not attainable for PHP projects, local dependencies are a must have. |
|
@alcohol: multiple version of PHP dependencies globally installed haven't been a problem on Gentoo up until the arrival of Composer. Beside this new problem of PHP and Composer, this applies to any language as well. Be it C/C++, Python or anything else. |
|
If you can back that statement up with an actual real-world use-case and show me how it is applied, I will believe it. |
|
@MarkKemp Bull... command-line is not that difficult a barrier. Quite frankly, if someone is trying to learn to program, they should become acquainted with it at some point. Earlier the better. |
|
@MarkKemp can't tell if serious, or troll. plz explain |
|
@sint I can't believe you when you are saying
But I'll be happy to understand how you are achieving that without having one web server of php-fpm per PHP application. EDIT: Or you are patching the applications with calls to |
|
Which make me terribly sad reading this thread, is that some PHP users think PHP stack may be different that any other language stack... sounds so wrong. E.G. why some standard, like semver, exists ? Unsubscribing as this is obviously going nowhere, Trollday ! |
|
@franzliedke Not too many right now. But if you force everyone to use Composer for every PHP program and library? (That's what's happening, inadvertently.) @rquadling If you install everything with a package manager, nothing ever breaks, because the package manager ensures that only satisfactory versions are installed together. If this sounds like beautiful magic, it is -- package managers are a godsend and that's why I'd like to be able to use mine. @alcohol This sounds like the "PHP is special" argument again. I don't know what to say to convince you that it isn't. "Software evolves faster..." is a self-fulfilling prophecy. If you bundle everything so that nothing has to work together because nothing works together because you bundled it all during development... well, you get what you get. But I'm getting off track; we have tens of thousands of packages that work just fine this way. We only package things that do actually work. If I take a Composer project and add back its Look at it this way: if it were possible to install a Composer project with my package manager and it still didn't work (for whatever reason), at least I wouldn't be here complaining about it. That would be my problem as the maintainer. I can be at peace with omitting some package that truly does not work or causes conflicts. But the barriers here are artificial and that's why it's frustrating. |
|
@remicollet it is different actually, different from your OS loader which is able to provide you different version of glibc depending on your needs. PHP just can't do that, the engine can't decide which version to use and the first file found matching the required name will be served. You can help/trick it with a custom PHP config, using function calls like |
|
@orlitzky please see remicollets links, he packages composer packages for Fedora without problems. You can use composer.json details to write an autoloader for every package including its dependencies and ship it under vendor/autoload.php of each package, this way it should work generically to convert any composer package $x into one or more gentoo packages (the application + all its dependencies). |
|
its my strongest believe, that every beginner should start at the command line, and the first time he runs a php script it should be by starting the script from the command line. Its even more my believe, that someone who starts using components by other people should have learned the basics first, as he otherwise won't be able to make an estimate of the cost and the risk involved of using someone elses code. |
|
OK everyone piling up here with nonsense about the OP and your beliefs about CLI: you are not helping. You are just spamming 20 people with an argument that was made. Everyone else on the system package manager debate: I think we have heard all arguments. If someone from Gentoo is interested in getting things working they can go check what remicollet did and then ask him or us any further questions if needed. Closing this now. |
You wanted a specific complaint? Here it is:
Why are beginners being herded into the command line, due to composer?
Application developers are now frequently only including composer installs.
This effectively locks beginners out of the learning process. It creates yet another barrier to entry, simply for the convenience of a minority.
The text was updated successfully, but these errors were encountered: