Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

patch an issue with AWS S3 SDK #3701

Closed

Conversation

sebartyr
Copy link

@sebartyr sebartyr commented Nov 30, 2022

patch an issue with AWS S3 SDK in src/Api/Serializer/ForumSerializer.php
SDK crashes without this patch and we can't use our S3-like buckets

patch an issue with AWS S3 SDK
SDK crashes without this patch and we can't use our S3-like buckets
@sebartyr sebartyr requested a review from a team as a code owner November 30, 2022 13:18
@sebartyr
Copy link
Author

This is our stack trace :
[2022-11-30 10:44:43] flarum.ERROR: InvalidArgumentException: Found 1 error while validating the input provided for the GetObject operation:
[Key] expected string length to be >= 1, but found string length of 0 in /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/Api/Validator.php:65
Stack trace:
#0 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/Middleware.php(87): Aws\Api\Validator->validate('GetObject', Object(Aws\Api\StructureShape), Array)
#1 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/S3/S3Client.php(589): Aws\Middleware::Aws{closure}(Object(Aws\Command), NULL)
#2 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/S3/S3Client.php(612): Aws\S3\S3Client::Aws\S3{closure}(Object(Aws\Command), NULL)
#3 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/S3/S3Client.php(546): Aws\S3\S3Client::Aws\S3{closure}(Object(Aws\Command), NULL)
#4 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/S3/S3Client.php(565): Aws\S3\S3Client::Aws\S3{closure}(Object(Aws\Command), NULL)
#5 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/Middleware.php(57): Aws\S3\S3Client::Aws\S3{closure}(Object(Aws\Command), NULL)
#6 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/S3/SSECMiddleware.php(59): Aws\Middleware::Aws{closure}(Object(Aws\Command), NULL)
#7 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/IdempotencyTokenMiddleware.php(77): Aws\S3\SSECMiddleware->__invoke(Object(Aws\Command), NULL)
#8 [internal function]: Aws\IdempotencyTokenMiddleware->__invoke(Object(Aws\Command))
#9 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/functions.php(363): call_user_func(Object(Aws\IdempotencyTokenMiddleware), Object(Aws\Command))
#10 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/aws/aws-sdk-php/src/S3/S3Client.php(509): Aws\serialize(Object(Aws\Command))
#11 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/illuminate/filesystem/FilesystemAdapter.php(521): Aws\S3\S3Client->getObjectUrl('cc-support-flar...', '')
#12 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/illuminate/filesystem/FilesystemAdapter.php(470): Illuminate\Filesystem\FilesystemAdapter->getAwsUrl(Object(League\Flysystem>
#13 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/flarum/core/src/Api/Serializer/ForumSerializer.php(95): Illuminate\Filesystem\FilesystemAdapter->url('')
#14 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/flarum/core/src/Api/Serializer/AbstractSerializer.php(88): Flarum\Api\Serializer\ForumSerializer->getDefaultAttributes(Array)
#15 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/sycho/json-api/src/Resource.php(183): Flarum\Api\Serializer\AbstractSerializer->getAttributes(Array, NULL)
#16 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/sycho/json-api/src/Resource.php(81): Tobscure\JsonApi\Resource->getAttributes()
#17 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/sycho/json-api/src/Document.php(188): Tobscure\JsonApi\Resource->toArray()
#18 /home/bas/app_9acb5f0b-33d9-4204-8762-c1f2a7e66d73/vendor/sycho/json-api/src/Document.php(232): Tobscure\JsonApi\Document->toArray()

We use Cellar from Clever Cloud with "league/flysystem-aws-s3-v3": "^1.0" dependency

Without this patch, we got "something went wrong" on our flarum's home page when we updated from 1.1 to 1.5 or 1.6

@luceos
Copy link
Member

luceos commented Dec 1, 2022

Hello @sebartyr,

Can you clarify how you instantiate the s3 adapter inside Flarum? I'm especially interested in the config params as the FilesystemAdapter seems to apply some of these values while generating the URL.

Thanks.

@sebartyr
Copy link
Author

Sorry for my late answer. Actually, i'm not working on it at the moment. I will let you know when i will be on it again

@sebartyr
Copy link
Author

@luceos

We use this class :

<?php

namespace CleverCloud\Flarum;

use Flarum\Filesystem\DriverInterface;
use Flarum\Settings\SettingsRepositoryInterface;
use Flarum\Foundation\Config;
use Illuminate\Contracts\Filesystem\Cloud;
use League\Flysystem\Filesystem;
use Aws\S3\S3Client;
use League\Flysystem\AwsS3v3\AwsS3Adapter as S3Adapter;
use Illuminate\Filesystem\FilesystemAdapter;

class CellarFilesystemDriver implements DriverInterface
{
    public function build(string $diskName, SettingsRepositoryInterface $settings, Config $config, array $localConfig): Cloud
    {
        $client = new S3Client($config['cellar']);

        $adapter = new S3Adapter($client, $config['buckets'][$diskName]);

        $filesystem = new Filesystem($adapter);

        return new FilesystemAdapter($filesystem);
    }
}

and in extend.php :

<?php

/*
 * This file is part of Flarum.
 *
 * For detailed copyright and license information, please view the
 * LICENSE file that was distributed with this source code.
 */

use Flarum\Extend;

return [
    (new Extend\Console())->command(\CleverCloud\Flarum\ServeCommand::class),
    (new Extend\ServiceProvider())->register(CleverCloud\Flarum\SyslogServiceProvider::class),
    (new Extend\Filesystem)->driver('cellar', CleverCloud\Flarum\CellarFilesystemDriver::class),
    (new Extend\Console)->command(Blomstra\CacheAssets\Command\CacheAssetsCommand::class),
    new Blomstra\Redis\Extend\Redis([
        'host' => env('REDIS_HOST', 'localhost'),
        'port' => env('REDIS_PORT', 6379),
        'password' => env('REDIS_PASSWORD'),
        'database' => 0
    ]),
    (new Extend\Validator(Flarum\User\UserValidator::class))->configure(CleverCloud\Flarum\RegistrationValidator::class),
];

@luceos
Copy link
Member

luceos commented Jan 25, 2023

Still missing the structure of the config you're using to instantiate the cellar driver then I guess?

@sebartyr
Copy link
Author

sebartyr commented Jan 25, 2023

'cellar' => [
        'endpoint' => 'https://' . env('CELLAR_ADDON_HOST'),
        'region' => 'us-east-1',
        'use_path_style_endpoint' => true,
        'version' => 'latest',
        'credentials' => [
            'key' => env('CELLAR_ADDON_KEY_ID'),
            'secret' => env('CELLAR_ADDON_KEY_SECRET'),
        ],
    ],
'buckets' => [
    'flarum-assets' => env('ASSETS_BUCKET'),
    'flarum-avatars' => env('AVATARS_BUCKET'),
    'flarum-uploads' => env('UPLOADS_BUCKET'),
]

This ? @luceos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants