[WIP] New asset management #4855

Merged
merged 15 commits into from Sep 17, 2014

Conversation

Projects
None yet
@qiangxue
Member

qiangxue commented Aug 28, 2014

Fixes #4454

Problems being addressed

Most css/js libs are not registered as Composer packages. As a result, if an extension has dependency on them, it becomes very troublesome to install the extension. As an example, the yii2-jui extension is including the whole JUI lib because there is no JUI composer package. And because of this, it is possible that two extensions are using different ways to reference the same 3rd party js lib and thus cause double inclusion of the same js file.

Current status

Basic app (including debugger and gii) can work running the following composer commands:

composer global require "fxp/composer-asset-plugin:1.0.*@dev"
composer update

Tasks to be done

  • support automatic bower/npm package installation when using composer to install an extension. This is the main blocking task currently. We plan to solve this with composer-asset-plugin.
  • enhance asset command to support asset bundle listing and asset combining/compression via grunt and other tools (may postpone this task to 2.0.1)
  • fix unit tests for asset bundles and asset manager
  • fix acceptance/functional tests for apps
  • documentation

Summary of changes

  1. Introduced @bower and @npm aliases to refer to the directories that should be used to install bower and npm packages, respectively.
  2. Removed 3rd party js libs (e.g. jquery ui, punycode) from the core codebase and rely on bower dependencies to install them when needed.
  3. Standardized the way of organizing assets, as described below.

Assets specific to application

Put the assets under a Web folder, and declare them in one or multiple asset bundle classes. For an example, see https://github.com/yiisoft/yii2/blob/new-asset/apps/basic/assets/AppAsset.php

Assets for non-redistributable widgets/modules

Put the assets under either a Web folder or the directory containing the widget/module class file. If the latter, make sure you specify AssetBundle::sourcePath so that the assets can be published when being used.

Assets for extensions

Put the assets in a directory (e.g. assets) under the extension base path. List them in a bundle class and make sure you set AssetBundle::sourcePath to be @vendor/PackageName/assets.

Third-party assets

If your application or extension uses a third-party asset package, do these steps:

  • Create an asset bundle to list the asset files to be used. Make sure you set AssetBundle::sourcePath to be @bower/PackageName, if the third-party package is a bower package (@npm/PackageName for npm packages, @vendor/PackageName for Composer packages).
  • In the asset bundle class for your application/extension, list the third-party bundle class as a dependency.
  • In composer.json of your application/extension, list the third-party package as a dependency.
@@ -89,16 +89,16 @@ following way:
```php
class LanguageAsset extends AssetBundle
{
- public $language;
+ public static $language;

This comment has been minimized.

@samdark

samdark Aug 29, 2014

Member

Why static?

@samdark

samdark Aug 29, 2014

Member

Why static?

extensions/bootstrap/composer.json
+ ],
+ "require": {
+ "yiisoft/yii2": "*",
+ "twbs/bootstrap": "3.2.* | 3.1.* | 3.0.*"

This comment has been minimized.

@samdark

samdark Aug 29, 2014

Member

Should be removed?

@samdark

samdark Aug 29, 2014

Member

Should be removed?

@@ -76,16 +76,19 @@ If you want to specify additional properties of the style tag, pass an array of
If you need to make sure there's only a single style tag use fourth argument as was mentioned in meta tags description.
```php
-$this->registerCssFile("http://example.com/css/themes/black-and-white.css", [BootstrapAsset::className()], ['media' => 'print'], 'css-print-theme');
+$this->registerCssFile("http://example.com/css/themes/black-and-white.css", [

This comment has been minimized.

@samdark

samdark Aug 29, 2014

Member

I like the change. Syntax is now more explanatory.

@samdark

samdark Aug 29, 2014

Member

I like the change. Syntax is now more explanatory.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

Why it was changed overall ? Does other fw use bower ? Looks like too much steps to start the demo app

Contributor

Ragazzo commented Aug 29, 2014

Why it was changed overall ? Does other fw use bower ? Looks like too much steps to start the demo app

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

Amount of steps will be the same when it will be bound to Composer.

Member

samdark commented Aug 29, 2014

Amount of steps will be the same when it will be bound to Composer.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

no, now i need to install nodejs and deal with it bugs and so on , or not ?

Contributor

Ragazzo commented Aug 29, 2014

no, now i need to install nodejs and deal with it bugs and so on , or not ?

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

It's extremely easy using both Windows, MacOS and Linux. Easier than installing Composer actually.

Member

samdark commented Aug 29, 2014

It's extremely easy using both Windows, MacOS and Linux. Easier than installing Composer actually.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

i am not sure what is overall profit ? only switching composer to bower ? because i dont see any advantages except everybody use bower, can you maybe specify them in particular ?

Contributor

Ragazzo commented Aug 29, 2014

i am not sure what is overall profit ? only switching composer to bower ? because i dont see any advantages except everybody use bower, can you maybe specify them in particular ?

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

It's all described in asset-related issues. But here's the summary of two main reasons:

  1. We're getting same path for the same library no matter which extension is using it. It makes situation with using two same JS or CSS files from different locations nearly impossible.
  2. It gives us a way to automate getting JS and CSS the same great way we're getting PHP packages. Composer doesn't fit there because nearly all the clientside library authors are using bower and not using Composer.
Member

samdark commented Aug 29, 2014

It's all described in asset-related issues. But here's the summary of two main reasons:

  1. We're getting same path for the same library no matter which extension is using it. It makes situation with using two same JS or CSS files from different locations nearly impossible.
  2. It gives us a way to automate getting JS and CSS the same great way we're getting PHP packages. Composer doesn't fit there because nearly all the clientside library authors are using bower and not using Composer.
@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

So now when creating extension everyone should mention it in packagist and in bower packages ? is not it too much ?

Contributor

Ragazzo commented Aug 29, 2014

So now when creating extension everyone should mention it in packagist and in bower packages ? is not it too much ?

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

Not if your extension uses third party JS instead of your own.

Member

samdark commented Aug 29, 2014

Not if your extension uses third party JS instead of your own.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

so bower actually installs all packages in some other directory, and in our extendsions and widgets we should somehow link to it with aliases ? should we create @assets or something like that in application to be able to register js ?

Contributor

Ragazzo commented Aug 29, 2014

so bower actually installs all packages in some other directory, and in our extendsions and widgets we should somehow link to it with aliases ? should we create @assets or something like that in application to be able to register js ?

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

Check the code.

Member

samdark commented Aug 29, 2014

Check the code.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

from where bootstrap and other comes ? I checked code but no clarification in docs or in other places .
Also how actually bower will help since you hardcoded assets like 'yii2-gii/assets/gii.js', in code ? how one can install something in yii2-gii since it is only for yii2 ?

Contributor

Ragazzo commented Aug 29, 2014

from where bootstrap and other comes ? I checked code but no clarification in docs or in other places .
Also how actually bower will help since you hardcoded assets like 'yii2-gii/assets/gii.js', in code ? how one can install something in yii2-gii since it is only for yii2 ?

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

@qiangxue can you clarify overall advantages ( just interesting ) ? and what advantage of replacing files is in particular ?

Contributor

Ragazzo commented Aug 29, 2014

@qiangxue can you clarify overall advantages ( just interesting ) ? and what advantage of replacing files is in particular ?

@pmoust

This comment has been minimized.

Show comment
Hide comment
@pmoust

pmoust Aug 29, 2014

Contributor

@Ragazzo Mark, it's a step forward, could you present any benefit in not using bower? A positive effect of sticking with previous situations, apart from we've been doing like so for a few years and it adds nodejs dependency.

Contributor

pmoust commented Aug 29, 2014

@Ragazzo Mark, it's a step forward, could you present any benefit in not using bower? A positive effect of sticking with previous situations, apart from we've been doing like so for a few years and it adds nodejs dependency.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

So what are real advantages and step forward is ? How it will solve problem with file conflicts and so on, since they will be named exactly as before ? We only switched composer to bower and thats it ? Also i am not against but want to see real advantages, because now it is only looks like a decision from holywar issue started by one of developers

Contributor

Ragazzo commented Aug 29, 2014

So what are real advantages and step forward is ? How it will solve problem with file conflicts and so on, since they will be named exactly as before ? We only switched composer to bower and thats it ? Also i am not against but want to see real advantages, because now it is only looks like a decision from holywar issue started by one of developers

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

Again, it was all discussed and answered:

Member

samdark commented Aug 29, 2014

Again, it was all discussed and answered:

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

@samdark the solution with

'class' => 'yii\web\AssetManager',
    'assetMap' => [
        // when seeing jquery.min.js, we should use jquery.js instead
        'bower:jquery/jquery.min.js' => 'bower:jquery/jquery.js',

        // when seeing anything matching */jquery.js, use the bower version only
        '*/jquery.js' => 'bower:jquery/jquery.js',
    ],
]

seems very hacky to me, all this unneeded prefixes and suffixes . Whatever overall

Contributor

Ragazzo commented Aug 29, 2014

@samdark the solution with

'class' => 'yii\web\AssetManager',
    'assetMap' => [
        // when seeing jquery.min.js, we should use jquery.js instead
        'bower:jquery/jquery.min.js' => 'bower:jquery/jquery.js',

        // when seeing anything matching */jquery.js, use the bower version only
        '*/jquery.js' => 'bower:jquery/jquery.js',
    ],
]

seems very hacky to me, all this unneeded prefixes and suffixes . Whatever overall

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Aug 29, 2014

Member

That's why current approach is different while it solves the same problems.

Member

samdark commented Aug 29, 2014

That's why current approach is different while it solves the same problems.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

Ok , thank you

Contributor

Ragazzo commented Aug 29, 2014

Ok , thank you

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Aug 29, 2014

Member

I am no expert at Bower, but as I see it behaves similar to Composer, placing all packages files under particular directory.
If it is so, why do we need this full-scale change? We can introduce a specific path alias named @bower or @assetsrc, which will point to the bower target directory. So all we need is specify AssetBundle::sourcePath to be @bower and specify JS and CSS relatively.
We may even mock-up bower running, composing its target directory manually for the core assets.

I don't lilke the idea of entire loosing ability to specify assets from extension local directory. This ability is very usefull while creating java script solutions, which are specific for the particular project. For example: I often create a custom widget to encapsulate complex catalog filter, which uses JavaScript and Ajax for funcy effects. During this process I create JQuery component named like 'itemFilter' and place it in the separated js file. It makes no sense for mwe to create a separated bower package to store it, because it is unlikely will be used anywhere else, but I still want my Widget to publish my js and include it to the page. Current fix makes this not possible.

Member

klimov-paul commented Aug 29, 2014

I am no expert at Bower, but as I see it behaves similar to Composer, placing all packages files under particular directory.
If it is so, why do we need this full-scale change? We can introduce a specific path alias named @bower or @assetsrc, which will point to the bower target directory. So all we need is specify AssetBundle::sourcePath to be @bower and specify JS and CSS relatively.
We may even mock-up bower running, composing its target directory manually for the core assets.

I don't lilke the idea of entire loosing ability to specify assets from extension local directory. This ability is very usefull while creating java script solutions, which are specific for the particular project. For example: I often create a custom widget to encapsulate complex catalog filter, which uses JavaScript and Ajax for funcy effects. During this process I create JQuery component named like 'itemFilter' and place it in the separated js file. It makes no sense for mwe to create a separated bower package to store it, because it is unlikely will be used anywhere else, but I still want my Widget to publish my js and include it to the page. Current fix makes this not possible.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

yes , i also support @bower or other alias since it is very flexible to use with widgets and extensions in asset bundles , but @klimov-paul as was said by @samdark it is already fixed, not sure where though, maybe he will do some clarification

Contributor

Ragazzo commented Aug 29, 2014

yes , i also support @bower or other alias since it is very flexible to use with widgets and extensions in asset bundles , but @klimov-paul as was said by @samdark it is already fixed, not sure where though, maybe he will do some clarification

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Aug 29, 2014

Member

Well, pardon we if I am wrong, but as see from the soure code of this PR AssetBundle::sourcePath is simply removed as well as entire publishing logic of AssetManager, which elminates the entire ability of publishing assets via PHP. I suppose PR leaves it all to Bower.

Member

klimov-paul commented Aug 29, 2014

Well, pardon we if I am wrong, but as see from the soure code of this PR AssetBundle::sourcePath is simply removed as well as entire publishing logic of AssetManager, which elminates the entire ability of publishing assets via PHP. I suppose PR leaves it all to Bower.

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Aug 29, 2014

Member

I updated the description of this PR trying to clarify some of your concerns. If you still have concerns, please feel free to keep discussing about them here. Yes, this change is very significant and may require change of your previous development habit, but I feel it is necessary and we are on the right direction

Member

qiangxue commented Aug 29, 2014

I updated the description of this PR trying to clarify some of your concerns. If you still have concerns, please feel free to keep discussing about them here. Yes, this change is very significant and may require change of your previous development habit, but I feel it is necessary and we are on the right direction

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

@qiangxue so what is with widgets ? If i dont want to register them on bower and so on, only internal project widgets usage, i should copy them into web folder as other for example web/assets/*my_widget*/css or web/assets/*my_widget*/js ?

Contributor

Ragazzo commented Aug 29, 2014

@qiangxue so what is with widgets ? If i dont want to register them on bower and so on, only internal project widgets usage, i should copy them into web folder as other for example web/assets/*my_widget*/css or web/assets/*my_widget*/js ?

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Aug 29, 2014

Member

@Ragazzo Yes.

If your widget is to be used in multiple private projects, the best practice is to create a composer package and a bower package and host them in a private repo. Otherwise, you have to manually copy them to the correct places.

If the widget is only developed and used for a single project, you should place the asset files under a Web folder directly.

Member

qiangxue commented Aug 29, 2014

@Ragazzo Yes.

If your widget is to be used in multiple private projects, the best practice is to create a composer package and a bower package and host them in a private repo. Otherwise, you have to manually copy them to the correct places.

If the widget is only developed and used for a single project, you should place the asset files under a Web folder directly.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Aug 29, 2014

Contributor

Ok , thanks for clarification, would be also great to note it ( that need widgets assets manually add under web folder or host private repo ) in docs or in description here

Contributor

Ragazzo commented Aug 29, 2014

Ok , thanks for clarification, would be also great to note it ( that need widgets assets manually add under web folder or host private repo ) in docs or in description here

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Aug 29, 2014

Member

Just updated my last comment to clarify this. Yes, all these will be added to the guide when the whole feature is ready.

Member

qiangxue commented Aug 29, 2014

Just updated my last comment to clarify this. Yes, all these will be added to the guide when the whole feature is ready.

@lubosdz

This comment has been minimized.

Show comment
Hide comment
@lubosdz

lubosdz Aug 29, 2014

Contributor

I am worriend about "Dropped asset publishing feature". Will it still be possible to publish in runtime any single JS/CSS file modified also during runtime?

(for example my app has built-in CSS online editor, so everytime user changes some CSS definition I need to save it into private directory to keep bundle files consistent and also instantly publish such a file with all saved changes).

Side note: from my experience, using bower makes sense if bower packages are intended for using by third party projects. E.g. developer may use all Yii2 widgets - is this the purpose ..? I am not really convinced it makes sense for Yii2 widgets/packages. Clearer use cases to justify this significant move would be nice. Using bower also puts extra knowledge (steeper learning curve) on all developers - they are forced to learn about node.js, how to use it, how to solve related issues ... I prefer keep it simple :-)

Contributor

lubosdz commented Aug 29, 2014

I am worriend about "Dropped asset publishing feature". Will it still be possible to publish in runtime any single JS/CSS file modified also during runtime?

(for example my app has built-in CSS online editor, so everytime user changes some CSS definition I need to save it into private directory to keep bundle files consistent and also instantly publish such a file with all saved changes).

Side note: from my experience, using bower makes sense if bower packages are intended for using by third party projects. E.g. developer may use all Yii2 widgets - is this the purpose ..? I am not really convinced it makes sense for Yii2 widgets/packages. Clearer use cases to justify this significant move would be nice. Using bower also puts extra knowledge (steeper learning curve) on all developers - they are forced to learn about node.js, how to use it, how to solve related issues ... I prefer keep it simple :-)

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Aug 29, 2014

Member

Will it still be possible to publish in runtime any single JS/CSS file modified also during runtime?

What asset publishing really does is merely file copying. This can be easily done without using asset manager, at least for the use case you are describing.

from my experience, using bower makes sense if bower packages are intended for using by third party projects...

If you are mainly extension consumers, you do not need to be aware of bower if you don't like to deal with it. If you are extension creators, you do need the knowledge about bower (and composer). They are the modern way of managing package dependencies.

Member

qiangxue commented Aug 29, 2014

Will it still be possible to publish in runtime any single JS/CSS file modified also during runtime?

What asset publishing really does is merely file copying. This can be easily done without using asset manager, at least for the use case you are describing.

from my experience, using bower makes sense if bower packages are intended for using by third party projects...

If you are mainly extension consumers, you do not need to be aware of bower if you don't like to deal with it. If you are extension creators, you do need the knowledge about bower (and composer). They are the modern way of managing package dependencies.

@tomfotherby tomfotherby referenced this pull request in mikehaertl/packagecompressor Aug 29, 2014

Closed

same scripts from dependences renders multiple times #12

@cebe

This comment has been minimized.

Show comment
Hide comment
@cebe

cebe Aug 29, 2014

Member

I'd prefer a solution that uses composer by default using https://github.com/francoispluchino/composer-asset-plugin and allows using bower if someone wants to use it. Will try to provide more detailed feedback next week.

Member

cebe commented Aug 29, 2014

I'd prefer a solution that uses composer by default using https://github.com/francoispluchino/composer-asset-plugin and allows using bower if someone wants to use it. Will try to provide more detailed feedback next week.

@lucianobaraglia

This comment has been minimized.

Show comment
Hide comment
@lucianobaraglia

lucianobaraglia Sep 1, 2014

Contributor

Hi!
Do this affects a theme development?
Let's say I just have a custom theme with custom css and js, do I still need Bower?
My Theme assets is like this:
app/themes/shadowbox/ShadowboxThemeAsset.php

<?php
namespace app\themes\shadowbox;

use yii\web\AssetBundle;

class ShadowboxThemeAsset extends AssetBundle
{

    public $sourcePath = '@app/themes/shadowbox/assets';

    public $css = [
        'css/custom.css',
        'http://fonts.googleapis.com/css?family=Raleway:400,500,700',
        'css/font-awesome.min.css',
        'css/framework.css',
        'css/settings.css',
        'css/style.css',
        //
    ];

    public $js = [
        '//s7.addthis.com/js/300/addthis_widget.js#pubid=xa-520049cc4619a30c',
        'http://maps.google.com/maps/api/js?sensor=false',
        'js/jquery.gomap-1.3.2.min.js',
        'js/jquery.themepunch.plugins.min.js',
        'js/jquery.themepunch.revolution.min.js',
        'js/isotope.js',
        'js/tabs.js',
        'js/touch-swipe.js',
        'js/ebor_carousel.js',
        'js/custom.js',
    ];

    public $depends = [
        'app\assets\DondeImprimoAsset',
    ];
}

Thanks!!!

Contributor

lucianobaraglia commented Sep 1, 2014

Hi!
Do this affects a theme development?
Let's say I just have a custom theme with custom css and js, do I still need Bower?
My Theme assets is like this:
app/themes/shadowbox/ShadowboxThemeAsset.php

<?php
namespace app\themes\shadowbox;

use yii\web\AssetBundle;

class ShadowboxThemeAsset extends AssetBundle
{

    public $sourcePath = '@app/themes/shadowbox/assets';

    public $css = [
        'css/custom.css',
        'http://fonts.googleapis.com/css?family=Raleway:400,500,700',
        'css/font-awesome.min.css',
        'css/framework.css',
        'css/settings.css',
        'css/style.css',
        //
    ];

    public $js = [
        '//s7.addthis.com/js/300/addthis_widget.js#pubid=xa-520049cc4619a30c',
        'http://maps.google.com/maps/api/js?sensor=false',
        'js/jquery.gomap-1.3.2.min.js',
        'js/jquery.themepunch.plugins.min.js',
        'js/jquery.themepunch.revolution.min.js',
        'js/isotope.js',
        'js/tabs.js',
        'js/touch-swipe.js',
        'js/ebor_carousel.js',
        'js/custom.js',
    ];

    public $depends = [
        'app\assets\DondeImprimoAsset',
    ];
}

Thanks!!!

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Sep 1, 2014

Member

In the current state of the advanced app I see that assets folder is linked to the frontend and backend applications. Thats great, but what if I don't want for backend js files would be published in the frontend?

As you can see, the assets path contains all assets used in the same project which may include different tiers. For development, this is fine. For production, you should select the needed js/css files, combine/compress them for each tier.

More over: maybe I have file called app.js both in backend and in frontend, how could I separate them?

The assets path is to hold 3rd party libs. You app.js should be put in a place specific for its application. So there won't be a conflict here.

Often I have different dirrectory structure for my development environment ...

It's not common to use different structures for different environments. But if you really do this, you need more customization to the init script. You should then consider using init to manage .bowerrc. I don't think we will do this by default.

Do this affects a theme development?

If your theme is a redistributable extension, you will have no problem because composer installer will publish the assets. If your theme is part of your application, you need to put the view code under protected folder while the assets in Web folder, because there's no more automatic asset publishing (if this is really important to you, you may enhance the Theme class to support automatic publishing.)

Member

qiangxue commented Sep 1, 2014

In the current state of the advanced app I see that assets folder is linked to the frontend and backend applications. Thats great, but what if I don't want for backend js files would be published in the frontend?

As you can see, the assets path contains all assets used in the same project which may include different tiers. For development, this is fine. For production, you should select the needed js/css files, combine/compress them for each tier.

More over: maybe I have file called app.js both in backend and in frontend, how could I separate them?

The assets path is to hold 3rd party libs. You app.js should be put in a place specific for its application. So there won't be a conflict here.

Often I have different dirrectory structure for my development environment ...

It's not common to use different structures for different environments. But if you really do this, you need more customization to the init script. You should then consider using init to manage .bowerrc. I don't think we will do this by default.

Do this affects a theme development?

If your theme is a redistributable extension, you will have no problem because composer installer will publish the assets. If your theme is part of your application, you need to put the view code under protected folder while the assets in Web folder, because there's no more automatic asset publishing (if this is really important to you, you may enhance the Theme class to support automatic publishing.)

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

At #4734 we have a long discussion about spliting application into a modules. What if I have the asset files dedicated to the particular module? I whould be unable to store them under the module directory with this change.

Member

klimov-paul commented Sep 2, 2014

At #4734 we have a long discussion about spliting application into a modules. What if I have the asset files dedicated to the particular module? I whould be unable to store them under the module directory with this change.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Sep 2, 2014

Contributor

@qiangxue What about the asset converter feature? I guess it will also be removed, right? (sorry, for not checking the PR, but it's a bit lengthy). If not, how will it fit into the picture?

Contributor

mikehaertl commented Sep 2, 2014

@qiangxue What about the asset converter feature? I guess it will also be removed, right? (sorry, for not checking the PR, but it's a bit lengthy). If not, how will it fit into the picture?

@cebe cebe added this to the 2.0 RC milestone Sep 2, 2014

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Sep 2, 2014

Contributor

@klimov-paul indeed, but i think we can make some console command like in Symfony or Laravel, like yii asset/register path/of/yours . Not a big problem overall , maybe it even should not be included in core and developers can make it on their own since it is simple copy

Contributor

Ragazzo commented Sep 2, 2014

@klimov-paul indeed, but i think we can make some console command like in Symfony or Laravel, like yii asset/register path/of/yours . Not a big problem overall , maybe it even should not be included in core and developers can make it on their own since it is simple copy

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

What about the asset converter feature?

It remains. Don't worry.

big problem overall , maybe it even should not be included in core and developers can make it on their own since it is simple copy

Yes it is a simple copy, just like Active Record is a simple wrap around PDO.
If you remember the origin of AssetManager, it was created to allow publishing of CSS and JS files, which are not stored under web directory. This feature allows binding CSS and JS files to the particular PHP code, which relates to them, like widget or module.
For me this is very common feature, and as I understand term 'framework' - it is a library, which provides common features, so I don't need to invent them myself.

Bower is suitable for handling common shared libraries like JQuery, Bootstrap and so on. But it does not suites for custom minor solutions, which I create every day composing custom pages with specific JS components.

Member

klimov-paul commented Sep 2, 2014

What about the asset converter feature?

It remains. Don't worry.

big problem overall , maybe it even should not be included in core and developers can make it on their own since it is simple copy

Yes it is a simple copy, just like Active Record is a simple wrap around PDO.
If you remember the origin of AssetManager, it was created to allow publishing of CSS and JS files, which are not stored under web directory. This feature allows binding CSS and JS files to the particular PHP code, which relates to them, like widget or module.
For me this is very common feature, and as I understand term 'framework' - it is a library, which provides common features, so I don't need to invent them myself.

Bower is suitable for handling common shared libraries like JQuery, Bootstrap and so on. But it does not suites for custom minor solutions, which I create every day composing custom pages with specific JS components.

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

In particular, look at yii\authclient\widgets\AuthCoice widget:
https://github.com/yiisoft/yii2/blob/master/extensions/authclient/widgets/AuthChoice.php

It has a simple JQuery component bound to it and CSS related to it. Does these files really worth to be composed into full-scale Bower package? Its look like usage of remote controlled robot for pushing buttons at the keyboard for you.

Member

klimov-paul commented Sep 2, 2014

In particular, look at yii\authclient\widgets\AuthCoice widget:
https://github.com/yiisoft/yii2/blob/master/extensions/authclient/widgets/AuthChoice.php

It has a simple JQuery component bound to it and CSS related to it. Does these files really worth to be composed into full-scale Bower package? Its look like usage of remote controlled robot for pushing buttons at the keyboard for you.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Sep 2, 2014

Contributor

@klimov-paul so other fw works fine and Yii2 can't or what is the reason to not move on, on this one ? We do not loose anything as for me, but removing too much very specific things in asset publishing , that overall does not have real benefits except publishing - copy from one folder to another, this can be done by console command. Not sure i got your example with AuthChoice can you clarify ? However i am for keeping asset bundles, just for moving this one from web to console if it can be said so

Contributor

Ragazzo commented Sep 2, 2014

@klimov-paul so other fw works fine and Yii2 can't or what is the reason to not move on, on this one ? We do not loose anything as for me, but removing too much very specific things in asset publishing , that overall does not have real benefits except publishing - copy from one folder to another, this can be done by console command. Not sure i got your example with AuthChoice can you clarify ? However i am for keeping asset bundles, just for moving this one from web to console if it can be said so

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

what is the reason to not move on

I have never told we should not move on. Usage of Bower (or its analog) is nice modern feature, which should be present - we have decided this already.

does not have real benefits except publishing - copy from one folder to another, this can be done by console command

At the present state this is done automatically. When I want ot use a widget, which requires specific asset file(s) I do not need any special actions - I just put it into the view code, and does everything else without me. With this change I have to run bower (or "specific console command") before I can really use a widget. I just imagine numerous complains about "widget XXX not working" we will receive afterwards.

Not sure i got your example with AuthChoice can you clarify ?

Here is JS related to this widget:
https://github.com/yiisoft/yii2/blob/master/extensions/authclient/assets/authchoice.js
It contains simple JQuery component, which I have written to manage popup windows opening.
It is relatively simple, unlikely to be used anywhere else and dedicated to the AuthChoice widget logic.
Using philosophy of this change we should create a Bower package to store 68 lines(!) of JS code. Or force a developer to copy AuthChoice asset files manually to web folder before actual usage.
While at present state developer does not know anything about this JS at all and does not bother about it anyhow.

Member

klimov-paul commented Sep 2, 2014

what is the reason to not move on

I have never told we should not move on. Usage of Bower (or its analog) is nice modern feature, which should be present - we have decided this already.

does not have real benefits except publishing - copy from one folder to another, this can be done by console command

At the present state this is done automatically. When I want ot use a widget, which requires specific asset file(s) I do not need any special actions - I just put it into the view code, and does everything else without me. With this change I have to run bower (or "specific console command") before I can really use a widget. I just imagine numerous complains about "widget XXX not working" we will receive afterwards.

Not sure i got your example with AuthChoice can you clarify ?

Here is JS related to this widget:
https://github.com/yiisoft/yii2/blob/master/extensions/authclient/assets/authchoice.js
It contains simple JQuery component, which I have written to manage popup windows opening.
It is relatively simple, unlikely to be used anywhere else and dedicated to the AuthChoice widget logic.
Using philosophy of this change we should create a Bower package to store 68 lines(!) of JS code. Or force a developer to copy AuthChoice asset files manually to web folder before actual usage.
While at present state developer does not know anything about this JS at all and does not bother about it anyhow.

@Ragazzo

This comment has been minimized.

Show comment
Hide comment
@Ragazzo

Ragazzo Sep 2, 2014

Contributor

Ah , i see your point , true and valid though

Contributor

Ragazzo commented Sep 2, 2014

Ah , i see your point , true and valid though

@U-D13

This comment has been minimized.

Show comment
Hide comment
@U-D13

U-D13 Sep 2, 2014

Maybe I'm getting something wrong, but creating a bower package for every asset bundle sounds really uncomfortable to me. In my opinion the previous pattern was more sound.

Also having to convince our systems architect, that to run Yii2 we need node.js installed, because asset management has to be done via bower — does not sound like much fun.

I would not argue, that the option to manage assets via bower is a great optional feature, but, in my opinion, it ought to remain an optional feature, not one of core requirements.

U-D13 commented Sep 2, 2014

Maybe I'm getting something wrong, but creating a bower package for every asset bundle sounds really uncomfortable to me. In my opinion the previous pattern was more sound.

Also having to convince our systems architect, that to run Yii2 we need node.js installed, because asset management has to be done via bower — does not sound like much fun.

I would not argue, that the option to manage assets via bower is a great optional feature, but, in my opinion, it ought to remain an optional feature, not one of core requirements.

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Sep 2, 2014

Member

@klimov-paul your use case is valid but only till the project goes production where you'll combine/compress all JavaScript to all.js and all CSS to all.css anyway. That was our initial thinking for removing runtime asset publishing.

For extensions we can add a special composer.json command that copies (or symlinks) assets at installation same way as it's now changing permissions. If there's a bower packages it could run bower command as well. It will solve a problem with immediate extension usage.

@U-D13 you can use http://bowerphp.org/ if your architect absolutely insists about avoiding node. The main reason for it is that if we'll use bower, extension authors will have proper dependencies so you won't end up with two jQueryUI included by two different extensions.

Member

samdark commented Sep 2, 2014

@klimov-paul your use case is valid but only till the project goes production where you'll combine/compress all JavaScript to all.js and all CSS to all.css anyway. That was our initial thinking for removing runtime asset publishing.

For extensions we can add a special composer.json command that copies (or symlinks) assets at installation same way as it's now changing permissions. If there's a bower packages it could run bower command as well. It will solve a problem with immediate extension usage.

@U-D13 you can use http://bowerphp.org/ if your architect absolutely insists about avoiding node. The main reason for it is that if we'll use bower, extension authors will have proper dependencies so you won't end up with two jQueryUI included by two different extensions.

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

your use case is valid but only till the project goes production where you'll combine/compress all JavaScript to all.js and all CSS to all.css anyway

Current implementation covers compoistion of the source files for such compression, which this PR removes.

For extensions we can add a special composer.json command that copies (or symlinks) assets at installation same way as it's now changing permissions. If there's a bower packages it could run bower command as well. It will solve a problem with immediate extension usage.

I am talking not only about extensions. I am talking about widgets and modules, which developer may create for the particular project AuthChoice is just an example of how such widget may look like. This change will enforce developer either to use inline scripts instead of *.js files, which will not allow further JS combining into "all.js", or drop the incapsulation and isolation of the particular widgets and modules creating shared throuhg entire application *.js files.

My opinion, with this PR we throwing ourthelves from one edge to another.

Member

klimov-paul commented Sep 2, 2014

your use case is valid but only till the project goes production where you'll combine/compress all JavaScript to all.js and all CSS to all.css anyway

Current implementation covers compoistion of the source files for such compression, which this PR removes.

For extensions we can add a special composer.json command that copies (or symlinks) assets at installation same way as it's now changing permissions. If there's a bower packages it could run bower command as well. It will solve a problem with immediate extension usage.

I am talking not only about extensions. I am talking about widgets and modules, which developer may create for the particular project AuthChoice is just an example of how such widget may look like. This change will enforce developer either to use inline scripts instead of *.js files, which will not allow further JS combining into "all.js", or drop the incapsulation and isolation of the particular widgets and modules creating shared throuhg entire application *.js files.

My opinion, with this PR we throwing ourthelves from one edge to another.

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Sep 2, 2014

Member

Any idea on how to take the best from both approaches while keeping extension authors from using common library JS/CSS from bower?

Member

samdark commented Sep 2, 2014

Any idea on how to take the best from both approaches while keeping extension authors from using common library JS/CSS from bower?

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

I have posted it at my first enty in this disscussion:
#4855 (comment)

We may created a specific alias named @bower or something, which will point to the Bower destination directory. At all asset bundles, which uses shared libraries we can use this alias to define actual JS and CSS files.

So JqueryAsset will change from:

class JqueryAsset extends AssetBundle
{
    public $sourcePath = '@vendor/yiisoft/jquery';
    public $js = [
        'jquery.js',
    ];
}

to:

class JqueryAsset extends AssetBundle
{
    public $sourcePath = '@bower/dist/jquery';
    public $js = [
        'jquery.js',
    ];
}
Member

klimov-paul commented Sep 2, 2014

I have posted it at my first enty in this disscussion:
#4855 (comment)

We may created a specific alias named @bower or something, which will point to the Bower destination directory. At all asset bundles, which uses shared libraries we can use this alias to define actual JS and CSS files.

So JqueryAsset will change from:

class JqueryAsset extends AssetBundle
{
    public $sourcePath = '@vendor/yiisoft/jquery';
    public $js = [
        'jquery.js',
    ];
}

to:

class JqueryAsset extends AssetBundle
{
    public $sourcePath = '@bower/dist/jquery';
    public $js = [
        'jquery.js',
    ];
}
@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

Better even use baseUrl:

class JqueryAsset extends AssetBundle
{
    public $baseUrl = '@bower/dist/jquery';
    public $js = [
        'jquery.js',
    ];
}

Since ClientScript already prevents including files with duplicated URLs - all problem solved.

Member

klimov-paul commented Sep 2, 2014

Better even use baseUrl:

class JqueryAsset extends AssetBundle
{
    public $baseUrl = '@bower/dist/jquery';
    public $js = [
        'jquery.js',
    ];
}

Since ClientScript already prevents including files with duplicated URLs - all problem solved.

@samdark

This comment has been minimized.

Show comment
Hide comment
@samdark

samdark Sep 2, 2014

Member

@qiangxue what do you think about what @klimov-paul proposes?

Member

samdark commented Sep 2, 2014

@qiangxue what do you think about what @klimov-paul proposes?

@U-D13

This comment has been minimized.

Show comment
Hide comment
@U-D13

U-D13 Sep 2, 2014

@samdark I was having a discussion with our sysarchitect right while I was writing that comment, and "there's also BowerPHP," was his exact words, "but it does not work properly". BowerPHP is alpha, not something one would want to rely on in a production environment.

U-D13 commented Sep 2, 2014

@samdark I was having a discussion with our sysarchitect right while I was writing that comment, and "there's also BowerPHP," was his exact words, "but it does not work properly". BowerPHP is alpha, not something one would want to rely on in a production environment.

@klimov-paul

This comment has been minimized.

Show comment
Hide comment
@klimov-paul

klimov-paul Sep 2, 2014

Member

@U-D13, you may compose your deployment process in a way production server will not call Bower.
You can compose bower destination folder content at the development server and then tranfer it to the production one. Thus production server may have no NodeJS and Bower installed.

Such problems can be solved.

Member

klimov-paul commented Sep 2, 2014

@U-D13, you may compose your deployment process in a way production server will not call Bower.
You can compose bower destination folder content at the development server and then tranfer it to the production one. Thus production server may have no NodeJS and Bower installed.

Such problems can be solved.

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Sep 2, 2014

Member

Yes, perhaps we should bring back asset publishing because based on the feedback, it seems many developers are relying on this feature implicitly or explicitly.

We may also need to put 3rd party asset libs under vendor and use asset publishing to copy only the needed libs to the Web folder. To ensure each asset lib has a unique URL, we should enforce a rule about the location of each asset lib, e.g., vendor/bower/PackageName (for bower packages), vendor/npm/PackageName (for npm packages). If an extension contains some assets that you absolutely do not need to be extended (e.g. AuthChoice), then its source path would be its composer package path.

Regarding the dependency on bower/npm, I have evaluated bowerphp. Unfortunately, it has a lot of issues and is not usable. So now we are counting on https://github.com/francoispluchino/composer-asset-plugin/, which based on its documentation is very promising. It will eliminate the requirement of installing nodejs/bower and making installing an extension just like a regular composer package.

Member

qiangxue commented Sep 2, 2014

Yes, perhaps we should bring back asset publishing because based on the feedback, it seems many developers are relying on this feature implicitly or explicitly.

We may also need to put 3rd party asset libs under vendor and use asset publishing to copy only the needed libs to the Web folder. To ensure each asset lib has a unique URL, we should enforce a rule about the location of each asset lib, e.g., vendor/bower/PackageName (for bower packages), vendor/npm/PackageName (for npm packages). If an extension contains some assets that you absolutely do not need to be extended (e.g. AuthChoice), then its source path would be its composer package path.

Regarding the dependency on bower/npm, I have evaluated bowerphp. Unfortunately, it has a lot of issues and is not usable. So now we are counting on https://github.com/francoispluchino/composer-asset-plugin/, which based on its documentation is very promising. It will eliminate the requirement of installing nodejs/bower and making installing an extension just like a regular composer package.

@rubenheymans

This comment has been minimized.

Show comment
Hide comment
@rubenheymans

rubenheymans Sep 3, 2014

This is awesome!

This is awesome!

@francoispluchino

This comment has been minimized.

Show comment
Hide comment
@francoispluchino

francoispluchino Sep 3, 2014

@creocoder for francoispluchino/composer-asset-plugin:

And main thing is this far from support of all bower features (.bowerrc, package cleanup after install, etc).

The package cleanup after install is supported for bower asset. Only bower script is not supported.

This solution will (probably) require publication since composer vendor is not under webroot.

For use the plugin in project mode (webroot), the PR composer/composer#3082 has a status 'feature' and should be merged.

Currently, the plugin has only 39 stars, which is low, but if you include the plugin in the framework, this digit will increase rapidly.

Of course I am open to any suggestions to improve the plugin.

@creocoder for francoispluchino/composer-asset-plugin:

And main thing is this far from support of all bower features (.bowerrc, package cleanup after install, etc).

The package cleanup after install is supported for bower asset. Only bower script is not supported.

This solution will (probably) require publication since composer vendor is not under webroot.

For use the plugin in project mode (webroot), the PR composer/composer#3082 has a status 'feature' and should be merged.

Currently, the plugin has only 39 stars, which is low, but if you include the plugin in the framework, this digit will increase rapidly.

Of course I am open to any suggestions to improve the plugin.

@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Sep 3, 2014

Member

@francoispluchino Thank you for your comment. I tried your plugin, and it worked great! I'm now eagerly looking forward to seeing its project mode support getting merged soon. This will make everything transparent to end users.

Member

qiangxue commented Sep 3, 2014

@francoispluchino Thank you for your comment. I tried your plugin, and it worked great! I'm now eagerly looking forward to seeing its project mode support getting merged soon. This will make everything transparent to end users.

+ $asset = $bundle->sourcePath . '/' . $asset;
+ }
+
+ $n = strlen($asset);

This comment has been minimized.

@samdark

samdark Sep 3, 2014

Member

Directory may contain non-latin characters so mb_strlen should be used instread of strlen.

@samdark

samdark Sep 3, 2014

Member

Directory may contain non-latin characters so mb_strlen should be used instread of strlen.

- $opts['afterCopy'] = $this->afterCopy;
- }
- FileHelper::copyDirectory($src, $dstDir, $opts);
+ if ($this->linkAssets) {

This comment has been minimized.

@samdark

samdark Sep 3, 2014

Member

I think we should make $linkAssets true by default. It is well supported by all modern file sytems and OSes.

@samdark

samdark Sep 3, 2014

Member

I think we should make $linkAssets true by default. It is well supported by all modern file sytems and OSes.

@cebe

This comment has been minimized.

Show comment
Hide comment
@cebe

cebe Sep 14, 2014

Member

@qiangxue have not read it but maybe interesting information: composer/composer#3078 (comment)

Member

cebe commented Sep 14, 2014

@qiangxue have not read it but maybe interesting information: composer/composer#3078 (comment)

qiangxue added some commits Sep 16, 2014

Merge branch 'master' into new-asset
Conflicts:
	apps/basic/composer.json
	composer.json
Fixes #4454: Improved asset bundle managed and used composer-asset-p…
…lugin to manage the dependencies on 3rd-party JS libraries

qiangxue added a commit that referenced this pull request Sep 17, 2014

@qiangxue qiangxue merged commit 999a39d into master Sep 17, 2014

1 check was pending

continuous-integration/travis-ci The Travis CI build is in progress
Details
@qiangxue

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Sep 17, 2014

Member

@francoispluchino FYI, yii2 master is now using your plugin to manage the asset package dependencies. So far it worked very well in my personal tests. Great job!

Member

qiangxue commented Sep 17, 2014

@francoispluchino FYI, yii2 master is now using your plugin to manage the asset package dependencies. So far it worked very well in my personal tests. Great job!

@samdark samdark deleted the new-asset branch Sep 17, 2014

@IndeedPlusPlus

This comment has been minimized.

Show comment
Hide comment
@IndeedPlusPlus

IndeedPlusPlus Sep 20, 2014

Contributor

I don't have such .bowerrc file and it just works fine. Should this file be ignored?

I don't have such .bowerrc file and it just works fine. Should this file be ignored?

This comment has been minimized.

Show comment
Hide comment
@cebe

cebe Sep 22, 2014

Member

this file has since been removed, no need for it.

Member

cebe replied Sep 22, 2014

this file has since been removed, no need for it.

This comment has been minimized.

Show comment
Hide comment
@qiangxue

qiangxue Sep 22, 2014

Member

Actually this file is still there. I kept it in case users want to work with bower directly in their projects.

Member

qiangxue replied Sep 22, 2014

Actually this file is still there. I kept it in case users want to work with bower directly in their projects.

@RomeroMsk

This comment has been minimized.

Show comment
Hide comment
@RomeroMsk

RomeroMsk Sep 22, 2014

Contributor

Please, add a note about depends key in options (registerJsFile/registerCssFile) to UPGRADE.md. It was surprising when assets stopped working after update.

Contributor

RomeroMsk commented on f51a263 Sep 22, 2014

Please, add a note about depends key in options (registerJsFile/registerCssFile) to UPGRADE.md. It was surprising when assets stopped working after update.

This comment has been minimized.

Show comment
Hide comment
@cebe

cebe Sep 22, 2014

Member

done 6c51866. Thanks for pointing it out.

Member

cebe replied Sep 22, 2014

done 6c51866. Thanks for pointing it out.

This comment has been minimized.

Show comment
Hide comment
@RomeroMsk

RomeroMsk Sep 22, 2014

Contributor

Thank you!

Contributor

RomeroMsk replied Sep 22, 2014

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment