Skip to content

support vcs on repos without composer.json #29

Open
c33s opened this Issue Jun 29, 2012 · 20 comments

10 participants

@c33s
c33s commented Jun 29, 2012

info: i use the yaml to json converter, so the following config parts are yml styled.

for example, if i want to include twitter bootstrap from the official repo, the following code works quite well:

repositories:
    # twitter/bootstrap
    - { type: package, package: { name: twitter/bootstrap, version: 2.0.0, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: v2.0.0 } } }
    - { type: package, package: { name: twitter/bootstrap, version: 2.0.1, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: v2.0.1 } } }
    - { type: package, package: { name: twitter/bootstrap, version: 2.0.2, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: v2.0.2 } } }
    - { type: package, package: { name: twitter/bootstrap, version: 2.0.3, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: v2.0.3 } } }
    - { type: package, package: { name: twitter/bootstrap, version: 2.0.4, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: v2.0.4 } } }
    - { type: package, package: { name: twitter/bootstrap, version: 2.1.0, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: origin/2.1.0-wip } } }

php bin/satis --verbose build config/config.json ../web/ produces a nice packages file

Scanning packages
Selected twitter/bootstrap (2.0.0)
Selected twitter/bootstrap (2.0.1)
Selected twitter/bootstrap (2.0.2)
Selected twitter/bootstrap (2.0.3)
Selected twitter/bootstrap (2.0.4)
Selected twitter/bootstrap (2.1.0)

but having

repositories:
    # twitter/bootstrap
    - { type: vcs, url: 'https://github.com/twitter/bootstrap.git' } #no composer

leads to

Reading composer.json of https://github.com/twitter/bootstrap.git (v1.2.0)
Skipped tag v1.2.0, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (v1.1.1)
Skipped tag v1.1.1, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (v1.1.0)
Skipped tag v1.1.0, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (v1.0.0)
Skipped tag v1.0.0, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (v1.4.0)
... snip ...
Reading composer.json of https://github.com/twitter/bootstrap.git (v2.0.2)
Skipped tag v2.0.2, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (2.1.0-wip)
Skipped branch 2.1.0-wip, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (gh-pages)
Skipped branch gh-pages, no composer file
Reading composer.json of https://github.com/twitter/bootstrap.git (master)
Skipped branch master, no composer file

so this is unhelpful. satis connects to the repo, reads the tag and uses it for an error message. why not generate

repositories:
    # twitter/bootstrap
    - { type: package, package: { name: twitter/bootstrap, version: %%calculated_version_number%%, source: { url: 'https://github.com/twitter/bootstrap.git', type: git, reference: %%tag_or_branch_composer_read%% } } }
    ...

this would really help and also really push composer. the only difficult here is to caclulate the %%calculated_version_number%% from the %%tag_or_branch_composer_read%%

where to add this code? satis is using composer, does satis generate this error meassage or is it composer? where to add what? if you can lead me the right direction i can try building a PR

@stof
stof commented Jun 29, 2012

The VCS repository is about telling composer to scan the VCS repo to read the metadata from the composer.json in each branch and tag. It cannot be used when you don't have a composer.json as the git repo does not contain enough information for composer. If your repository does not have a composr.json, you have to use the package repository

@c33s
c33s commented Jun 29, 2012

@stof to clearify what i mean:
i know that this is currently not possible, but satis/composer does the following:

  • take the url my own composer.json file provides (for example https://github.com/twitter/bootstrap.git)
  • it connects to the url and scans all tags and branches for composer.json files
  • if no composer.json file is found -> generate error message. this message contains the tag/branch name

this is the way i add a non-composer.json repo manually in my satis repo:

- { type: package, package: { name: $name, version: $version, source: { url: '$url', type: git, reference: $reference } } }

i have to repeat this step an manually change the $reference according the git repo tags/branches and also set the version manually.

so we have the $url and we have the $reference (tag/branch name), we can easily parse the $name from the $url, the only "difficult" thing is the $version but it should be not that difficult to generate a versionnumber from $reference, so why not generating a package out of it?

echoing an error message containing nearly all of the $vars i need is not helpful for the users of satis, adding a setting to like "autogenerate packages on non existing composer.json" and building such packages automatically makes much more sense.

do you know what i mean?

@stof
stof commented Jun 29, 2012

the only things that could be guessed if you were using the VCS repo withotu a composer.json is the version number. But this is the best way to generate broken packages. A composer package have much more settings than a version, and none of them can be guessed for a repo without a composer.json: http://getcomposer.org/doc/04-schema.md

@c33s
c33s commented Jun 29, 2012

nothing else as the version number is needed. the rest is provided by the url. nothing is broken if i know that its a auto generated repo. i am not talking about adding it this way to packagist but having to add every twitter/bootstrap version by hand is pain.
can't you see the benefit? just compare adding each version of twitter bootstrap by hand, watching their repo and again add a manual repo if a new version is there.
it's much more convenient having it auto added by satis. what should be broken than?

composer & packagist are not the "save-me-from-broken-packages" solution. a few days ago my composer.json of a symfony project was broken just because a official packagist package removed a version. it was simply gone.

so what is the exact problem of letting satis generate the repos for (on my demand, on a setting where i say "yes, please do that for me"), instead of making me add it line by line, by copy and paste? copy and paste is not error safe. it is much more reasonable that a human makes an error by manually adding the repos in the satis package config, then letting satis create it for me.

yes i know the schema definition of composer, but do i need it for simply adding twitter bootstrap, modernizr or font awesome?
they depend on nothing, they just should find their way in the vendor directory. nothing more, nothing less

@stof
stof commented Jun 30, 2012

@c33s nothing else is needed in the case of twitter bootstrap. For other packages, you may need to provide extra stuff. And the issue is that if satis starts auto-guessing packages, it would also do it for packages that would need extra informations, thus producing broken packages. Satis cannot know that this particular package is a special case.

@c33s
c33s commented Jun 30, 2012

@stof and in case of every other javascript libary, also in case of everything i manually add. satis should only autoguess packages where i want it. its an option, a possibility. don't fear RAD.
you can't save users from doing stupid thing. i makes no sense to strip down features and rad only because of fear something could happen.

it's like, no we shouldn't use symfony, a user can use plain php just in the app.php or in the php-template.

don't sacrifice RAD and nice features which you can control only because of something could break.

if haven't said that the repo type vcs should always guess. it should ONLY guess if i define something like:

repositories:
    # twitter/bootstrap
    - { type: vcs, url: 'https://github.com/twitter/bootstrap.git', autogenerate_on_missing_composerjson: true}
@till
till commented Jul 3, 2012

I think it would help adoption a lot, if we could consume packages the way @c33s suggests.

I understand that this would not give you the benefit of tracking dependencies and what not, but who cares? If you would end up using a package from a repo without a composer.json, you would have to take care of that and IMHO that is understood.

@schmunk42

I have the same request :)

Auto-generating packages.json would be really helpful for many use cases.
I've created a test script which converts the info you can get from the github API into a packages.json file. So far only license and dependencies are missing, see https://gist.github.com/3853414

@staabm
staabm commented Apr 24, 2013

would love to have this

@marcj marcj referenced this issue in composer/composer May 6, 2013
Closed

Repositories/Dependencies without composer.json #1870

@lavoiesl
lavoiesl commented Oct 8, 2013

+1

@jkobus
jkobus commented Dec 17, 2013

you can solve this by creating the "package" inside "repositories" in your composer.json.
Something like here:
https://github.com/phpextra/php-code-browser

@c33s
c33s commented Dec 17, 2013

@jkobus can you give an example please?
see my initial post, there i have defined each version i want to support as own package which is simply awkward if i run composer an see that all the data to autogenerate such repository definitions is there.
for me its not DRY and to manually enter all the data is quite error-prone

@schmunk42

@c33s Without a composer.json file in the repo you're loosing eg. the feature to define dependencies, autoloading, etc ... for sure it's not needed for twitter/bootstrap, but you also have to "guess" the package name, i.e. - there are simply too many things you can't detect 100% automatically.

What would be nice though, is a separate repo with legacy packages, which are created without the need of a composer.json file. I did this for my own Yii-specific project here: http://phundament.com/en/packages-12.html - but it needs some work, for the basic idea see my comment above.

@c33s
c33s commented Dec 17, 2013

@schmunk42 i know the features i loose, but if you add a repo like this:

- { type: vcs, url: 'https://github.com/twitter/bootstrap.git', autogenerate_on_missing_composerjson: true}

you know what you do and the feature is just helpful. nothing more nothing less.

@schmunk42

I think too many thing could go wrong here.
As for your example above the URL has changed, it's now "twbs/bootstrap", so what would be the autogenerated package name?

@parsingphase

For what it's worth in this specific case, twitter bootstrap has a composer.json file as of about a month ago, although this is obviously only useful for the most recent version:

https://github.com/twbs/bootstrap/blob/master/composer.json

However, beyond this it does seem utterly daft risky at best to tell composer to guess at information; installing packages that don't support it is well outside composer / satis' responsibility.

That said; we do have a use case for this where I work and have built a tool to scan git repos and generate packagelist.json files from well-formed (semver-compatible) tags; I'm hoping I can get permission to open-source this, but one thing we have found in using it is that there aren't even that many well-tagged git repos to use it with.

@jkobus
jkobus commented Dec 17, 2013

Sorry for necroposting. As I said it is possible to create package description without the need to modify the repository. So you actually can, if repository author is lazy or does not like composer for some reason, do it yourself.

You NEED to set-up your own satis repository (or any kind of repository as this is just a json file) and there you can add package definition:

{
    "name": "My satis",
    "homepage": "http://localhost.dev/satis/",
    "directory": "dist",
    "repositories": [
        { "type": "vcs", "url": "..." },
        { "type": "vcs", "url": "..." },

        {
            "type": "package",
            "package": {
                "name": "twatter/bootstrap",
                "version": "0.1.0",
                "source": {
                    "url": "git://github.com/twitter/bootstrap.git",
                    "type": "git",
                    "reference": "4dae89749c7301898756e0737c3f0a342602eff0"
                },
                "autoload": {
                    "classmap": [
                        "src"
                    ]
                }
            }
        },
         {
            "type": "package",
            "package": {
                "name": "twatter/bootstrap",
                "version": "0.1.1",
                "source": {
                    "url": "git://github.com/twitter/bootstrap.git",
                    "type": "git",
                    "reference": "origin/master"
                },
                "autoload": {
                    "classmap": [
                        "src"
                    ]
                }
            }
        }
    ],
    "require-dependencies": true,
    "require-all": true
}

You can also add the "package" directly to your composer.json and it will work with it.
This works exactly as you wanted in your first post.
Same applies to packages using pear without native composer support etc.

@c33s
c33s commented Dec 18, 2013

@schmunk42 its only about having a handy tool and knowing the risk.

if i for example add plain & ugly php code in the app.php of symfony2 i know that it is not clean and i risk that something will break, but sometimes it is just handy to add the code there.

so if the repo name changes, it will break in both ways, if satis generates the repository entries itself or if have written them myself.
if i have written them myself i have to fix the url in all repos i added. if they would have changed the version numbers (just for example) i have the add alll the versions again by myself.
if everything is autogenerated, i just fix one url. done.

satis reads the info from a vcs. it shows the info on the command line. i ask for writing it to the json file in the right format or just dump it to the command line and let me review and copy & paste it to the json file.

@parsingphase it is not about bootstrap, but there are some libs out there containing css/html/js which i just want to easyload. without overhead, without defining each version by myself. and sometimes it is even handy for a php lib.

@jkobus that is quite equivalent to the code i posted in my first post. in your post you add the repo multiple times (for each version you want to add, one time). this ticket is about not adding it multiple times and fill out all the detailed config.

@ all
i really don't see a risk here. i can review the code generated, i can change it, i can hopefully dont add new code directly on the production server but on my local test environment.

i personally think it is quite interesting that so many persons argue against a quite handy feature which just generates me plain json data which i easily can review. where really nothing risky is in it.
on the other side, so many symfony bundles can only be required with dev-master. where you have to pin point them on there sha1 hash, so you wont end up with a version which really changed its behavior.

@parsingphase

We've now released the previously-mentioned tool to scan git repos for versions and output the results as a satis.json file: https://github.com/FitchLearning/satisfy

Hopefully this will cover the needs for a few of the above use-cases.

@jrobeson
jrobeson commented Aug 9, 2014

i think the best solution is to use bower directly or perhaps this plugin: https://github.com/francoispluchino/composer-asset-plugin

and close this ticket as it seems like it will not be implemented.

@alcohol alcohol added the feature label Nov 21, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.