Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Allow (Re-)Naming of Pretty Name for a Repository #1355

Closed
mikeschinkel opened this Issue Nov 24, 2012 · 9 comments

Comments

Projects
None yet
3 participants

Currently if I repo has a composer file with the name "foo/bar" and config.vendor-dir = "packages" then the repo's files will be stored in "packages/foo/bar." However, in some cases it would be preferred to stored them in "packages/foo" or "packages/bar." It would be nice if when you specify a repo you could give it an alternate name for what directory it will be stored in.

Here's a StackOverflow question that details the use-case: http://stackoverflow.com/questions/13537161/

Contributor

stof commented Nov 24, 2012

Use a custom installer for this: http://getcomposer.org/doc/articles/custom-installers.md

Btw, for many frameworks, a custom installer already exists in https://github.com/composer/installers

@stof Thanks. Yes, a custom installer seemed to be what I needed but after 4 hours of trying to figure it out I was still unable to have any idea how to make it work. The docs simply don't provide enough examples and/or specifics.

That said, something as simple as changing the directory name shouldn't require a custom installer, should it?

@stof I just realized after posting the comments that the docs page you linked was not a page I had seen before, and I searched all over for docs on custom installers. Still, reading it I'm not exactly sure how to make it work, or maybe I'm just too brain dead after >12 hours of coding. Anyway, thanks for the links.

Owner

Seldaek commented Nov 24, 2012

@mikeschinkel it requires a custom installer because where code is installed in most cases should not matter. However most older CMSs and frameworks have restrictions in effect which is the reason we have this composer/installers project (as I wrote on stackoverflow before I saw this). Granted the docs may not be complete in that regard, if you need help feel free to ask here or on SO and if you find time to improve docs once you get it working it'd be great.

@Seldaek Seldaek closed this Nov 24, 2012

@Seldaek I'm sadden to hear you're unable to appreciate that for some people where code is installed really does matter. The CMSes you refer to have simplified directory structures because those directory structures are more applicable for their use-case. It's disappointing to learn that Composer has an attitude about such things.

Since you asked about my questions on the docs, here goes and please don't shoot the messenger.

One of the biggest problems is structural. For example, this document on custom installers has no links I can from from the main document page so I never found it until @stof linked it. I still don't know where I'd go to find it if I didn't have the link (besides Google.)

Further when on a page in the handbook there is no indication that it is part of a handbook; no reference to chapter or section, no sidebar of related topics, and no way to explore further without going back to the main document page and then drilling back down to find another related document. Yes, there are some in-document links but most web pages have in-document links so having them isn't the same has having navigation for a handbook.

Beyond the navigation issues, there's no walk-thru tutorial. I thought I'd find one in Getting Started but that just tells how to install Composer, nothing more. Very little discussion for how to use to compose on an existing project with no indication of what to expect and/or the problems someone might run into and no tutorial walking through the various commands that Composer supports with examples showing how they work. Most importantly no tutorial showing how to start from scratch to add composer to an existing project.

As for issues with specific examples, here's the document I found on How do I install a package to a custom path for my framework?. It calls out Package Authors and explains how to use "wordpress-theme" as a "type" but leaves the reader hanging when it comes to custom installer types; no mention of what to do if the type that you need isn't in the pre-existing list.

As for Package Consumers the example there is not clear either. It seems to be saying that this only works for composers installers but I'm left wondering why no "type":"library". Since it doesn't seem like a reasonable limitation and since I'm struggling to figure out a solution I waste time and try to get it working anyway.

But even if it had been clear that I definitely couldn't use "installer-paths" for custom installers I was stumped by the value in the example: ["vendor/package"]. Do I use "vendor/package" literally, or is that a placeholder? What exactly do I put there? The left side ("sites/example.com/modules/{$name}") is obviously a literal example so maybe the right side is too? And then I wonder if it is an array and if so what values do I put there? If there had been a comprehensive example with "real" hypothetical module names it would have probably resolved my confusion. As is it is just very unclear and makes it harder for people to adopt Composer and they might simply choose not to or just build what they need themselves. (Note that after studying many hours I think I know what "vendor/package" means but I didn't for a while. More examples can almost never hurt.)

Mosts of the example on the other pages seems to follow the same pattern; they provide examples that either do not provide enough context or that substitute abstract values in the examples rather than "real" hypothetical values (i.e. "vendor/package" rather than "acme/widget")

And on top of that, after reading all the docs I still don't know where or how to host my custom installer if I were to build one and how to get it to interact with my project's composer.json files. Looking at this document I'm told to specify "type"="composer-installer" but then I'm told to specify a type which my composer-installer defines. How can I specify two types at once? The example does not show how or where I would use "type"="composer-installer" so I'm stuck on that one.

Moving on it seems that "type"="composer-installer" is actually for Creating an Installer, but I'm still not sure. I see the structure of the composer.json file, but it's not clear to me how the autoload is related to this. I assume this installer would be hosted in a repo somewhere so wouldn't Composer handle the autoloading of this? And if a class has multiple installers, how are they used with one installer name? And how do I test one of these? I'm left at a loss for what to do next. It would be helpful to this document cover the complete process required to get one of these working, and even show an example of one that is working, maybe even at github.com/composer/custom-installer-example

If I could summaries all the above into two points it would be:

  1. The handbook needs better navigational aids,
  2. The handbooks should provide complete and testable examples, ideally ones that reference real repos so the exact examples can be tried and learned from.

Hope this helps, and again please don't shoot the messenger.

P.S. See, wouldn't it be 1000% easier for the Composer user to just be able to specify a "name" with a repository if they want to rename the directory that it gets installed in?

Owner

Seldaek commented Nov 25, 2012

@Seldaek I'm sadden to hear you're unable
to appreciate that for some people where code is installed really does
matter. [snip] It's disappointing to learn that Composer has an attitude
about such things.

Of course I can appreciate it, that's why we brought in the
composer/installers project in the composer organization. The problem is
composer can not know about every project out there, so we do what we
can to support major frameworks/CMSs in the composer/installers project,
and for the others you are welcome to send a PR. In this case though,
wordpress is fully supported as far as I know, so I don't know why you
complain that we ignore your needs.

While I don't intend on shooting the messenger for the message, I have
to say that the messenger does not have a very helpful attitude. I could
have done without reading rants that I closed the issue both here and on
SO. As I explained the stuff is supported, and as such this issue has no
reason to be left opened. It doesn't mean you can't still comment and
ask for help, I even welcomed you to do so.

One of the biggest problems is structural. For example, this document on
custom installers
http://getcomposer.org/doc/articles/custom-installers.md has no links
I can from from the main document page http://getcomposer.org/doc/

Under Articles the second link points exactly to that Custom Installers
page.

Further when on a page in the handbook there is no indication that it is
part of a handbook; no reference to chapter or section, no sidebar of
related topics, and no way to explore further without going back to the
main document page and then drilling back down to find another related
document.

Good point, though it's not so easy to address.

Beyond the navigation issues, there's no walk-thru tutorial. I thought
I'd find one in Getting Started http://getcomposer.org/doc/00-intro.md
but that just tells how to install Composer, nothing more.

At the end of the page there is a link to the next page, and that is
true for all the page that are part of the "Book" section. It is meant
to be a walk-thru of sorts. Maybe it's not clear enough?

Most importantly no
tutorial showing how to start from scratch to add composer to an
existing project.

Well the composer docs are mostly geared towards creating a new composer
file, adding your requirements and whatnot. If you are working in an
existing project that does not support composer (and I gotta say most
framworks and CMSs are moving towards that.. wordpress being one of the
notable exceptions as usual) you are a bit on your own. If you are
interested you can do like others did, encourage the leads of the WP
project to support it a bit better, or write a custom installer or
submit stuff to composer/installers if it's not complete yet. Get
involved. I personally can't hand-hold the entire php community through
the process, but I always try to help when people try to integrate it
into another project.

As for issues with specific examples, here's the document I found on How
do I install a package to a custom path for my framework?
http://getcomposer.org/doc/faqs/how-do-i-install-a-package-to-a-custom-path-for-my-framework.md.
It calls out Package Authors and explains how to use
|"wordpress-theme"| as a |"type"| but leaves the reader hanging when it
comes to custom installer types; no mention of what to do if the type
that you need isn't in the pre-existing list
https://github.com/composer/installers#current-supported-types. [snip]

Good point, I'll add some more content and cross-linking there. That
page definitely could use some love.

Mosts of the example on the other pages seems to follow the same
pattern; they provide examples that either do not provide enough context
or that substitute abstract values in the examples rather than "real"
hypothetical values /(i.e. "vendor/package" rather than "acme/widget")/

Good point as well, this is clear to people in the know but probably not
so for newcomers.

And on top of that, after reading all the docs I still don't know where
or how to host my custom installer if I were to build one and how to get
it to interact with my project's |composer.json| files. Looking at this
document http://getcomposer.org/doc/articles/custom-installers.md I'm
told to specify |"type"="composer-installer"| but then I'm told to
specify a type which my composer-installer defines. How can I specify
two types at once? The example does not show how or where I would use
|"type"="composer-installer"| so I'm stuck on that one.

The supports() method of the custom installer defines which types it
accepts, this is indeed not very clear in the docs. As for hosting it,
well you can put it on packagist or in your own repository, it's just a
package like any other: see http://getcomposer.org/doc/05-repositories.md

Moving on it seems that |"type"="composer-installer"| is actually for
/Creating an Installer/, but I'm still not sure. I see the structure
of the |composer.json| file, but it's not clear to me how the autoload
is related to this.

It is for creating one, and composer.json defines the autoloading for
every package, so that once your custom installer package is installed,
it can be autoloaded.

And if a
class has multiple installers, how are they used with one installer
name? And how do I test one of these?

One custom installer package can only have one class that is used as a
custom installer, but obviously that class can be a decorator to many
other installers/classes, much like the composer/installers project does
it. I don't think most installers would require this though.

P.S. See, wouldn't it be 1000% easier for the Composer user to just be
able to specify a |"name"| with a repository if they want to rename the
directory that it gets installed in?

Not really no. In most cases you don't define a repository, you just get
stuff from packagist or other private repositories. And in most cases
you also do not need to change the name. Either it's a plugin for a
framework/CMS and there is a specific place it should go, or it's a
generic library and nobody should care where it is since composer will
just autoload its classes. What you are suggesting would push the burden
of knowing this to every composer user out there, instead of having
package authors define it once and that's it.

The whole reason we use libraries is that doing the work once as a
library and then using it many times makes sense. Same goes for the
package and autoload definition.

Cheers

Jordi Boggiano
@Seldaek - http://nelm.io/jordi

In this case though, wordpress is fully supported as far as I know, so I don't know why you complain that we ignore your needs.

I'm working with WordPress but doing something non-standard with WordPress, in hopes, frankly to get enough people to use it so the core team wants to make it becomes a standard part of WordPress. Your custom installer handles the patterns I need but not the specifics.

Also, your WordPress custom installer is for website projects. I'm not building WordPress websites, I'm building and packaging WordPress plugins, which is a different project type with different requirements.

Under Articles the second link points exactly to that Custom Installers page.

I see that now. Not sure why I didn't see it before. It's not like I didn't spend all day Friday working on it. :(

At the end of the page there is a link to the next page, and that is
true for all the page that are part of the "Book" section. It is meant
to be a walk-thru of sorts. Maybe it's not clear enough?

Not all pages have that link. And since most pages are long it's hard to see it. Minimally it would help to have those links on the top of the page too. Beyond that some additional context-specific navigation would be really helpful. I do understand it's not trivial to add, my reason to comment is to let you know what someone who is not expert on Composer struggles with.

Well the composer docs are mostly geared towards creating a new composer
file, adding your requirements and whatnot.

That is exactly what I was trying to do. I was starting from scratch, and had a lot of difficulty.

If you are interested you can do like others did, encourage the leads of the WP
project to support it a bit better, or write a custom installer or submit stuff to composer/installers if it's not complete yet.

I would be interested, but I have to understand Composer first and make sure it can actually address the things that are important to WordPress people. BTW, it's only my opinion but I really think that this ticket is the type of thing that would be important to WordPress people. As an FYI, I was one of the first moderators on StackExchange's WordPress Answers though lately I've not had enough time to keep it up but that means I do know a bit about what WordPress developers in general think are important.

Good point as well, this is clear to people in the know but probably not so for newcomers.

Agree. When you are expert at something it's hard to even comprehend what someone new struggles with. I was a professional programming trainer and course material author for 7 years. During that time I learned a lot about what students struggled with in programming languages and my comments about the docs are in part based on my teaching experience.

It is for creating one, and composer.json defines the autoloading for
every package, so that once your custom installer package is installed,
it can be autoloaded.
One custom installer package can only have one class that is used as a
custom installer, but obviously that class can be a decorator to many
other installers/classes, much like the composer/installers project does
it. I don't think most installers would require this though.

I'm still stuck. I know it will be super simple when I see how it's done but I'm just stumped. If I saw a simple example that isn't one of the ones that ships with Composer then I think I could get it.

Not really no. In most cases you don't define a repository, you just get
stuff from packagist or other private repositories.

Maybe you misunderstood my original request? My use-case was for creating a composer.json file that would specify where it's dependencies are stored. Maybe it would help you to see the actual shell commands I'm using? These are the commands I run to "prepare" a plugin for testing. I pull down a BitBucket repo and then gather the dependencies with Composer and then clean it up so I can test it and later submit to WordPress SVN repository:

    cd ~/Sites/wp34/public_html/wp-content/plugins/
    hg clone https://mikeschinkel@bitbucket.org/mikeschinkel/my-plugin
    cd my-plugin
    composer install
    rm composer.json
    rm composer.lock
    rm .hgtags
    rm -Rf assets
    rm -Rf .hg
    rm -Rf libraries/restian/.git
    rm -Rf libraries/sidecar/.git
    rm -Rf libraries/composer/
    rm libraries/autoload.php
    rm libraries/restian/composer.json
    rm libraries/sidecar/composer.json
    #Do stuff to package plugin for SVN at wordpress.org

And in most cases you also do not need to change the name. Either it's a plugin for a
framework/CMS and there is a specific place it should go, or it's a generic library and
nobody should care where it is since composer will just autoload its classes.

In my case I'm trying to define a plugin development framework for WordPress and have strong opinions for how I want the directory structure to be in the framework. WordPress developers are notoriously finicky about avoiding anything that appears "too complex" and so I'm trying to keep it as simple as possible, including the directory structure. (I know you have an opinion that it doesn't matter but all that I ask is that you allow me to have an opinion that it does matter without dismissing my opinion as unimportant. That was why I "ranted", because I felt that you were not disrespecting my opinion. It's okay if you choose not to support it but please don't be disrespectful.)

What you are suggesting would push the burden of knowing this to every composer user out there, instead of having package authors define it once and that's it.

That doesn't sound like my use-case. Now I think you misunderstand what I wanted even more? Let's simplify it: I am building Plugin "A". It is using Libraries "B" and "C". They libraries can have composer files or not, that's fine. But I want to build a composer file for my plugin "A" so that I can embed Libraries "B" and "C" within my distribution, and I want to be able to specify the directory structure within "A".

Maybe understanding the WordPress ecosystem will help. The WordPress world doesn't do things like the rest of the PHP world. They rarely use autoloaders. Plugins are managed from wordpress.org not Packagist and plugins have headers that define information, but they don't manage dependencies. WordPress also has Themes where are similar to Plugins but you can only run one theme at a time. Most wouldn't know a psr if it bit them in the ass.

To date Plugins and Themes are pretty much all self-contained because there is no dependency management like there is with Modules in Drupal. Commerical theme vendors ship themes with functionality better suited for plugins because it's too much trouble for end users to understand how to manage plugins for their themes. And most plugins are fully self-contained too; there is almost nobody who distributes plugins or themes with any dependencies so I am using dependency management as a build-time tool, not a distribution tool.

PHP developers use PHP libraries that Composer can manage, but WordPress plugins and themes have to be able to be managed by end-users which is one reason why they need to be much simpler to deal with.

The whole reason we use libraries is that doing the work once as a
library and then using it many times makes sense. Same goes for the
package and autoload definition.

Unfortunately as I've explained, WordPress doesn't work exactly like that. Ironically though I am actually trying to change that. I've developed some libraries that I want to get people to use to build their plugins but until WordPress adds plugin dependency management into core plugins and themes will still need to be packaged as self-contained entities.

So what I want to use Composer for os to host plugins on GitHub or BitBucket that have Composer files that reference other repos on GitHub or BitBucket (or maybe later a WordPress-specific Packagist) and have a build process that takes all that dependency management, uses it and then thows it away so plugins can be committed to WordPress' SVN as self-contained collections of code.

FYI, I've literally just finished building a run-time dependency manager for WordPress that handles the situation when multiple plugin happen to include the same library because I'll need it for this.

Hopefully you can understand why my use-cases better, and maybe even want to help me address my struggles in hopes my work can get more WordPress developers using Composer?

Owner

Seldaek commented Nov 25, 2012

OK as usual it's easier to discuss a problem when we all know the goal. Now don't get this the wrong way but it sounds kind of insane to me what you are trying to do. I understand that it may be the most effective approach to "trojan-horse" composer into the wordpress community. But it would be way way easier if this was done in wordpress itself instead of having plugins/themes bundle their dependencies, and then have some intricate system to load a library while php provides you with autoloading already.

What I think would be a better (or at least more future-proof) approach is to just use composer as you would normally in your plugin, and let it put stuff in libraries/foo/bar or even vendor/foo/bar. Bundle all that, and then in the plugin bootstrap (I guess you have that somehow, I really don't know much about WP internals) require the autoload file if it is present. That way you'd end up with one autoloader per plugin that's done in this fashion, which is not ideal but then again if they each have only a couple libs it shouldn't be too wasteful. That way you avoid your whole run-time dependency manager, and require_library() and whatnot. You can just use classes autoloaded as anyone would. Then whenever WP core realizes that something outside of their ecosystem is happening and maybe they want to be part of it, they can use composer to install the plugins that are composer aware, and those plugins will suddenly get their dependencies installed in /root-wp-folder/vendor/... but that won't matter since they use autoloading already.

That should allow you to "composerify" existing plugins right now, while benefiting from a core comport support if it ever happens. Is this crazier?

The thing is there is like 0.001% chance that WordPress will adopt Composer for use with Plugins and Themes. They already have things that Composer would duplicate, they have a very significant amount of infrastructure in place that leverages those things and major change is not something the WordPress core team embraces.

For example there has been discussion for probably 3 years about moving to Git from SVN and there are many vocal people on the team who are very much against it. I expect they will move to Git eventually, but probably not for at least another 2 years. And moving to Git is so much less a change than moving to Composer for Plugins and Themes, I just don't see it happening. Ever. (To be clear, I have no say in those decisions so no need to convince me of the benefits cause it will just be wasted effort.)

Further, it's simply not workable to have Composer manage dependencies for Plugins and Themes. Plugins and Themes need to be able to be installed from shared hosting which would mean that each plugin would have to have a copy of Composer and all the hosts would have to have Git and maybe Hg installed. That's just not something shared web hosts make available. Remember, it is not developers that are installing these things from the command line, it is end users installing them from a web interface. So it's not even viable to have plugins published on the WordPress repository to have their dependencies managed by Composer; the best we can get is to use it for build management.

Something else I think you might be missing is that a WordPress site can have many plugins, each of which can potentially contain one of more libraries, some of which can be compatible and some of which can be incompatible. Unless WordPress used Composer to manage plugin dependencies they it doesn't work to use Composer for this (see my 0.001% comments above.) The require_library() functionality I wrote is designed to manage these issues in a manner that will be compatible as long as I never change the interface.

My hope is that WordPress will adopt require_library() and then it will be more robust. But to get there I have to build something that works well and then get lots of people to adopt it; they must believe my approach is simpler and easier than anything they can build on their own. If enough people adopt what I'm doing (and hopefully contribute to it) some form of it might finally find it's way to WordPress core. But even that will be a long road.

As you know WordPress has Plugins and Themes as first class concepts, but does not recognize the concept of a "Library"; I'd like to see that change. It's as if WordPress has Hammers and Screwdrivers but not Wrenches. Some things you can do with a hammer and others with a screwdriver but when you have a problem that needs a wrench having hammers and screwdrivers just won't do. I'd like to see WordPress add wrenches (i.e. libraries) and in open source advocate by building and then getting others to adopt what we build.

BTW, as for one auto-loader per plugin I cannot envision how that would work. WordPress developers use a lot of functions that cannot be autoloaded which kinda makes autoloading moot. In WordPress I find that autoloading only makes sense in a minority of cases and when it does it's usually highly dependent on class factory code.

Bottom line is that developing in WordPress is just extremely different from developing in an object-based MVC framework and most of the conventions developed by the PHP community just don't work (well?) with WordPress; WordPress still need 5.2.4 compatibility, for example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment