Add support for installing libraries via asset-packagist

[fago]

Let's support adding libraries via asset-packagist.

If the setup would be done by drupal-project, adding libraries would be simple. We are using this approach for adding module-needed libraries via composer for a while at drunomics now and it works great. It's been adopted by thunder 2.x as well, see https://github.com/BurdaMagazinOrg/thunder-project/pull/5/files.

Once this is done, you can add libraries via a simple command:

composer require bower-asset/dropzone=~4.2

More explanation: https://asset-packagist.org/site/about

[rodrigoaguilera]

I have been using it also and I think is great.
I copy the comment from here
#215

I found a way that can make the installation of external libraries from npmjs.org or bower into the folder /web/libraries/ easier

Recently this composer repo was created
https://asset-packagist.org/
but there still the problem that this libraries do not set a "type" for "composer/installers" to use so is difficult to set a path like /web/libraries/$name

There is this plugin that add a default type to all packages
https://github.com/oomphinc/composer-installers-extender

So we modify this project to add the assets repo, the plugin and write something like the following in the composer.json

"installer-types": ["npm-asset", "bower-asset"],
"installer-paths": {
  "web/libraries/{$name}": [
    "type:drupal-library",
    "vendor:npm-asset",
    "vendor:bower-asset"	
  ],
}

So we can do things like
composer require "npm-asset/jquery.easing":"~1.0"

and it will place the library into /web/libraries/jquery.easing

There is another explanation here
https://asset-packagist.org/site/about

No need to duplicate js libraries in packagist for a drupal "version".
In projects with a lot of external libraries the repositories section can get quite messy and https://github.com/fxpio/composer-asset-plugin is slow and complicated.
Also overcomes the limitations referenced in this PR about changing the library version.

Is it interesting to add this to the drupal composer project?

[arh1]

Great -- thanks.

To restate the steps above more explicitly:

1. Add the Asset Packagist repository to composer.json :

    "repositories": [
...
        {
            "type": "composer",
            "url": "https://asset-packagist.org"
        }
    ],

2. Add the Composer Installers Extender plugin:

composer require oomphinc/composer-installers-extender

3. Add installer types and paths to composer.json :

    "extra": {
...
        "installer-types": ["npm-asset", "bower-asset"],
        "installer-paths": {
...
            "your/installation/path/{$name}": [..., "type:npm-asset", "type:bower-asset"],
...
        }
...
    }

4. Add your npm or Bower packages from Asset Packagist, e.g.:

composer require npm-asset/select2

With the installer path above, select2 should now be installed to your/installation/path/select2

[rodrigoaguilera]

Exactly.

Now I extended this pattern to all my drupal 8 projects and it is working great.

I feel is time to do a PR with the changes explained here plus a little explanation in the README based on
#215

[ao2]

If we assume that polluting the root composer.json is OK, then a solution without a central repository could also be adding a "package" repository snippet directly to the root composer.json file to download the external library, defining the Drupal assets in hook_library_info_build.

However, IMHO, if a module depends on some external asset there should be a way to specify that dependency in a way local to the module (e.g. confined to the module composer.json or .info.yml) without leaking this info into the root composer.json, which would look like a layering violation to me (the main project knowing too much about the dependencies of its dependencies). Composer does not make this easy tho, because repositories can only be specified in the root composer.json file, see https://getcomposer.org/doc/faqs/why-can%27t-composer-load-repositories-recursively.md

The libraries module and the Libraries registry project try to solve that without the need to change the root composer.json file.

[rodrigoaguilera]

You have really good points but building a libraries registry sounds like setting up another niche repository of libraries useful for Drupal forgetting all that composer can bring you with semver and easy management.

However, IMHO, if a module depends on some external asset there should be a way to specify that dependency in a way local to the module (e.g. confined to the module composer.json or .info.yml)

I completely agree but there is no solution that allows you to manage the asset dependencies of your module from composer.

The libraries module and the Libraries registry project try to solve that without the need to change the root composer.json file.

I think is easier to submit the asset to packagist and define the dependency on the composer.json of your local module.

So from my point of view polluting the root composer.json is a trade-off for having an easier way of specifying asset dependencies for composer drupal projects and getting the flexibility of downloading them not only on "web/libraries"

[ao2]

You have really good points but building a libraries registry sounds like setting up another niche repository of libraries useful for Drupal forgetting all that composer can bring you with semver and easy management.

I am not a huge fan of the Libraries Registry either, however one point about it is that the format used for library definitions describes the assets from a Drupal point of view, freeing the module developer from having to make the assets accessible to the Drupal rendering system via hook_library_info_build() or MODULE.libraries.yml, this solves a Drupal problem and reduces code duplication.

However, IMHO, if a module depends on some external asset there should be a way to specify that dependency in a way local to the module (e.g. confined to the module composer.json or .info.yml)

I completely agree but there is no solution that allows you to manage the asset dependencies of your module from composer.

Yeah, composer not supporting recursive repositories is a major pain point.

I thought about a local solution for the libraries module in https://www.drupal.org/node/2887738 I'll try to come up with a proof-of-concept patch about it.

The libraries module and the Libraries registry project try to solve that without the need to change the root composer.json file.

I think is easier to submit the asset to packagist and define the dependency on the composer.json of your local module.

So from my point of view polluting the root composer.json is a trade-off for having an easier way of specifying asset dependencies for composer drupal projects and getting the flexibility of downloading them not only on "web/libraries"

If this was going to be the “official” way, adding support for package types from asset-packagist.org directly to composer/installers might be worth a try, this would make it possible to avoid composer-installers-extender and minimize the pollution of the main composer.json file.

I would still miss a solution without a centralized asset repository tho.

Anyway, an opinion from Drupal core developers could be useful on these matters.

[hawkeyetwolf]

I posted these comments on the PR but they should live here:

---

Adding component into the installer-types and library path definition would also add support for all the packages in the components org on Packagist proper, like components/chosen (for Chosen module), components/jquery_ns_autogrow (for Autogrow Textarea), and components/Sortable (for Paragraphs v1.2).

---

The asset-packagist approach doesn't work for contrib modules right now because they can't safely require asset-packagist dependencies. Per the composer docs:

Repositories are only available to the root package and the repositories defined in your dependencies will not be loaded. Read the FAQ entry if you want to learn why.

And this causes two problems when a contrib module requires an asset-packagist dependency:

1. Breaks testbot, since she doesn't have the asset-packagist repository definition in her root project (right now).
2. Prevents requiring the module via Composer by any site which doesn't have the asset-packagist repository defined. We can put big bold words on the d.o project page and in the README of each project, but it still seems problematic.

What do you think? Is there any way around this?

[fago]

What do you think? Is there any way around this?

Yeah, standardization & documentation. If we could standardize on using asset-packagist (or an alternative on d.o. infrastructure) this would be a non-issue.

[digitaldonkey]

This would be amazing. It would release the pressure of creating a update of a Drupal module every time a library packed changed, which currently makes using js assets practically unmaintainable.

[mxr576]

Related PR is here: #286

[rodrigoaguilera]

PR rebased and tests passing.

I just noticed more distributions are adding the asset-packist repo so is really becoming a standard already.
https://www.drupal.org/project/varbase/issues/2974100

One solution that hasn't been mentioned is that maybe the npm-asset and bower-asset vendors can be integrated somehow into the Drupal packagist repo
https://packages.drupal.org/8
after all is just free software and is published at
https://packagist.org/packages/hiqdev/asset-packagist
And since this libraries will be for only for Drupal they can have the drupal-library type already to simplify the root composer.json.

This will involve the Drupal association to do a significant amount of work but I feel it will be pretty ideal :)

[mxr576]

fxpio/composer-asset-plugin#330 if this PR gets accepted this could provide a better way not just for us, but for other PHP frameworks to depend on JS libraries (assets) without depending on the source of the library.

This is also related to hiqdev/asset-packagist#100.