Option to store files offsite

[tridoan]

Hello,
This is a feature request for enhancement.
Would be great to have an option to store uploaded files to AWS S3 (or comparable service). This is especially handy because it alleviates the need to upgrade a server just to get more storage space.

Thanks for providing this application!

[ganigeorgiev]

It is not documented very well, but this should be already supported.

1. Update the baseStorageUrl in your params-local.php

    // base public url to the storage directory (could be also a cdn address if you use S3 or other storage mechanism)
    'baseStorageUrl' => 'https://example.com/storage',

2. Update the fs component to use the S3 adapter in your base-local.php:

'components' => [
    ...
    'fs' => [
        'class'  => 'creocoder\flysystem\AwsS3Filesystem',
        'key'    => 'your-key',
        'secret' => 'your-secret',
        'bucket' => 'your-bucket',
        'region' => 'your-region',
        // 'version'  => 'latest',
        // 'baseUrl'  => 'your-base-url',
        // 'prefix'   => 'your-prefix',
        // 'options'  => [],
        // 'endpoint' => 'http://your-url'
    ],
    ...
]

For more option, you could check https://github.com/creocoder/yii2-flysystem.

[tridoan]

Thanks for this...but i couldnt get this working for some reason and there are no errors so i cant tell what the issue is. This is what i have:

params-local.php:
'baseStorageUrl' => 'https://folder-name.s3.amazonaws.com',

base-local.php:

        'awss3Fs' => [
            'class' => 'creocoder\flysystem\AwsS3Filesystem',
            'key' => 'myKEYHERE',
            'secret' => 'mySECRET',
            'bucket' => 'folder-name',
            'region' => 'us-west-1',
            // 'version' => 'latest',
            // 'baseUrl' => 'your-base-url',
            // 'prefix' => 'your-prefix',
            // 'options' => [],
            // 'endpoint' => 'http://my-custom-url'
        ],

I've also tried with 'fs' instead of 'awss3Fs' in base-local.php, but still no go.

I've confirmed that the keys/secret works and has access using the Cyberduck client, and was able to connect and write.

Is there something wrong with my syntax or am i missing some dependencies?

[ganigeorgiev]

@tridoan In order to override the base setting, the component key must be fs. If the credentials are correct, then I'm not sure what the issue may be.

Are the uploaded files still stored locally?

Could you post the network response when you try to upload an image (could be checked by navigating to the Network tab in the browser inspect tool)?

You could also enable the debug mode and change the api environment to dev by updating YII_DEBUG and YII_ENV constants in your api entrypoint (aka. web/index.php).

[ganigeorgiev]

I also forgot to mention that the AWS adapter is not included by default in the flysystem lib, so you'll need to run composer require league/flysystem-aws-s3-v3 to add it as a dependency in your project.

[tridoan]

Thanks, i think the dependency was part of the problem. I pulled down the dependency from composer as you stated just now. Enabled debug and dev environment.

Upon uploading an image, i see this error within XHR from inspect tools:

{"message":"Setting unknown property: creocoder\\flysystem\\AwsS3Filesystem::path","errors":{}}

[ganigeorgiev]

@tridoan Oops, I forgot that by default the base.php and base-local.php are recursively merged (see web/index.php).

To force replacing the fs component configuration, you will need to wrap it with \yii\helpers\ReplaceArrayValue:

'components' => [
    ...
    'fs' => new \yii\helpers\ReplaceArrayValue([
        'class'  => 'creocoder\flysystem\AwsS3Filesystem',
        'key'    => 'your-key',
        'secret' => 'your-secret',
        'bucket' => 'your-bucket',
        'region' => 'your-region',
    ]),
    ...
]

[tridoan]

Success! Kind of...
So with the wrapping of \yii\helpers\ReplaceArrayValue. this now successfully allows files to be written to S3, i've confirmed the files are being written to the bucket. Thanks for that.

My only issue now is that reading the files from S3, so that the thumbnails and images are visible within Presentator. As of now, the images are 403 Forbidden. I think this is a permissions issue on S3, so i need to play around with that a bit and see what i can do. But if you know offhand, that would be great.

Thanks again!

[ganigeorgiev]

@tridoan I'm glad that the upload works. As for the files access - you may want to check https://docs.aws.amazon.com/AmazonS3/latest/user-guide/block-public-access-bucket.html.

Side note: Depending on your usage, accessing s3 objects directly without a cdn sometimes could results in 503 errors because of throttling - https://docs.aws.amazon.com/AmazonS3/latest/dev/optimizing-performance.html.

[tridoan]

Thanks. I've figured it out, the instructions from AWS is incomplete. I also needed to update the Bucket Policy per this guide to allow for files to be readable publicly: https://havecamerawilltravel.com/photographer/how-allow-public-access-amazon-bucket/

Everything works now, thanks again!