AssetManager problem in cloud platform

[kamarton]

What steps will reproduce the problem?

Yii2 asset manager specified to any file storage and file serving. There is a small problem with this in a cloud platform.

  server A          server B            server C   ....     server X
     |                 |                   |                   |
     \---------------------------------------------------------/
                                |
                               LB
                                |
                          client browser

Yii2 schema:

1. client make request
2. server A
3. asset managerer copy or symlink files on server A
4. response
5. client make new request with asset urls
6. server B, C (LB select random servers)
7. not found asset files (onlye copy files server A)

Idea:

• It would be nice to have an easy-to-implement solution to not direct file serving.
• serving asset contents
  • from action (read files from direct path, without copy to asset folder)
  • support CDN PUT or another storage (redis, database)

What is the expected result?

files serving any servers

What do you get instead?

not found asset files (onlye copy files server A)

Additional info

yii2 solution (with my source codes):

1. asset manager put base directory to databse
2. asset serving action
3. get base directory from database
4. read and serving content from base directory + url path (with file checks and more)

Q	A
Version	2.x
PHP version	-
Operating system	-

[SamMousa]

I have built this: https://github.com/SAM-IT/yii2-static-assets

Things that are reusable for this use case include:

• ReadOnlyAssetManager, a manager that calculates the URL but doesn't do any publishing.
• AssetHelper, a helper class that allows you to find asset bundles in your code base and publish them using an asset manager of your choice.

https://github.com/SAM-IT/yii2-static-assets/blob/master/src/helpers/AssetHelper.php#L241

This could be used to publish to a directory on S3 during build.

[mikk150]

I am using https://gitlab.com/mikk150/yii2-asset-manager-flysystem

I had same issue, but even kind of worse, because I do not know(or care) about servers or how many of them I have(auto scaling applications based on load, by purchasing cheapest one from amazon, azure or digitalocean by bidding).. This is uploading them to CDN and CDN deals with it's thing

[terabytesoftw]

@kamarton @mikk150 @SamMousa @samdark @rustamwin @roxblnfk I have been working with assets, and that problem with flysystems is effectively eliminated, since it works with relative routes and part of it does not work with symlinks, now just working with the local adapter would be completely resolved, they think it is worth implementing that you can work with two adapters at the same time, example:

$mountManager->copy('local://sourcePath/file.ext', 'publish://basePath/file.ext');

Where local can be the local adapter and publish could be local, aws, digital ocean, google cloud storage, or any other adapter.

On the other hand, the other version problem can be solved by adding at the end of the hash 'v0' -> default version, and it can be customized in the assetBundle 'v1' ... 'v99', example hashv0 or (hashv1 ... hashv99).

[SamMousa]

I'm not sure how this is relevant @terabytesoftw, I'm assuming the flysystem asset manager already publishes data to any flysystem filesystem; how would having support for multiple help?

Note that there are already composite adapters that will allow you to write to multiple filesystems (local & remote) and then read with fallback (local first, remote otherwise):
https://flysystem.thephpleague.com/docs/adapter/replicate/
https://github.com/Litipk/flysystem-fallback-adapter

I'm not sure how having a mount manager that copies between paths is relevant here.
(I think I'm missing some context)

[samdark]

This is valid and serious problem that could be solved in multiple ways @SamMousa and @mikk150 suggested:

1. By sharing assets by writing them somewhere else like CDN or S3 bucket using https://github.com/yiisoft/yii-filesystem
2. By pre-collecting assets and publishing these during deployment. In this case we need to ensure that all servers have assets at the same path.
3. By pre-collecting assets and publishing these during development. Ready assets are then deployed to the servers.

Related #24 may greatly simplify asset list assembly for 2 and 3.

[SamMousa]

I think for the future we should do pre publishing. Moving to containers and immutable deploy artifacts is always safer. Even when wanting to deploy assets to a cdn doing so in advance is better.

[kamarton]

This is valid and serious problem that could be solved in multiple ways @SamMousa and @mikk150 suggested:

I like the idea.

By pre-collecting assets and publishing these during deployment. In this case we need to ensure that all servers have assets at the same path.

An interesting question here is whether it is always known in advance or not the path. Forces the deploy to receive a different route (path) from the previous state in each case.

For example, if the asset files of one package have not changed for 2 years, it will destroy the CDN function in an intensive development status by e.g. forces downloads to mobile devices up to several times a day.
This may not be a problem in the process, but developers need a guide in this area, otherwise they may run into this problem more easily.

[SamMousa]

For example, if the asset files of one package have not changed for 2 years, it will destroy the CDN function in an intensive development status by e.g. forces downloads to mobile devices up to several times a day.
This may not be a problem in the process, but developers need a guide in this area, otherwise they may run into this problem more easily.

You could still use the file hash as input for path generation, doing it in advance is no different from doing it at runtime..

[samdark]

I've recently wrote down about how to solve it in Yii 2: https://rmcreative.ru/blog/post/yii-2-za-balansirovschikom-nagruzki It is in Russian but may be relevant.

[samdark]

Seems to be possible with current master. Need to document it.

[samdark]

This one is now solved. An example added to readme.