S3 configuration

[FredPraca]

With Lychee v7.7.1, I use the following configuration for S3 storage

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: storage-env
data:
  S3_ENABLED: true
  AWS_ACCESS_KEY_ID: XXXXXXXX
  AWS_SECRET_ACCESS_KEY: XXXXXXXXX
  AWS_ENDPOINT: s3.fr-par.scw.cloud
  AWS_BUCKET: my-lychee-photo-storage
  AWS_DEFAULT_REGION: fr-par

But I also set a local storage based on the following comment.

        volumeMounts:
          - name: photo-storage
            mountPath: /app/public/uploads
          - name: app-storage
            mountPath: /app/storage/app
          - name: logs
            mountPath: /app/storage/logs
          - name: tmp
            mountPath: /app/storage/tmp

      volumes:
        - name: photo-storage
          hostPath:
            path: /var/www/Lychee-k8s/temp-photo-storage
        - name: app-storage
          hostPath:
            path: /var/www/Lychee-k8s/app-storage
        - name: logs
          hostPath:
            path: /var/www/Lychee-k8s/logs
        - name: tmp
          hostPath:
            path: /var/www/Lychee-k8s/tmp

The problem is that, on startup, I get the following exception:

a193dda984eb PHP Fatal error:  Uncaught RuntimeException: A facade root has not been set. in /app/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php:360
a193dda984eb Stack trace:
a193dda984eb #0 /app/app/Exceptions/Handler.php(435): Illuminate\Support\Facades\Facade::__callStatic()
a193dda984eb #1 /app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(641): App\Exceptions\Handler->report()
a193dda984eb #2 /app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(200): Illuminate\Foundation\Console\Kernel->reportException()
a193dda984eb #3 /app/artisan(35): Illuminate\Foundation\Console\Kernel->handle()
a193dda984eb #4 {main}
a193dda984eb   thrown in /app/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php on line 360

Removing the S3 part allows to startup flawlessly.

Is there something I didn't see in the configuration ?

[ildyria]

I am not using s3 so that one is difficult to answer. :/
But this facade root has not been set is a sign of a 500 happening before anything in Lychee. I smell missing dependency somewhere. :(

[FredPraca]

I'll take a look at the PR which included S3 support or maybe ask the contributor.

[ildyria]

https://laravel-news.com/using-aws-s3-for-laravel-storage
https://laravel.com/cloud/docs/resources/object-storage
also possibly interesting.

[FredPraca]

I'm used to use S3 as storage in several projects so creating buckets and so on is a quite easy task.
so I tried to insert S3 dependencies according to the articles you sent to see if new dependency would appear but apart from updating the version, no new dependency spawned.

@Kovah Any idea ?

[illhammf]

The first thing I would check is the value of AWS_ENDPOINT.

Your current configuration appears to use:

AWS_ENDPOINT: s3.fr-par.scw.cloud

For Scaleway Object Storage in the fr-par region, try using the complete endpoint URL, including the protocol:

AWS_ENDPOINT: "https://s3.fr-par.scw.cloud"

I would also recommend writing all values inside ConfigMap.data as strings. For example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: storage-env
data:
  S3_ENABLED: "true"
  AWS_DEFAULT_REGION: "fr-par"
  AWS_ENDPOINT: "https://s3.fr-par.scw.cloud"
  AWS_BUCKET: "my-lychee-photo-storage"
  AWS_USE_PATH_STYLE_ENDPOINT: "false"

The access key and secret key should preferably be stored in a Kubernetes Secret rather than directly inside the ConfigMap.

For example:

apiVersion: v1
kind: Secret
metadata:
  name: storage-secret
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: "your-access-key"
  AWS_SECRET_ACCESS_KEY: "your-secret-key"

Then reference both the ConfigMap and Secret from the Lychee deployment:

envFrom:
  - configMapRef:
      name: storage-env
  - secretRef:
      name: storage-secret

After updating the configuration, restart the deployment:

kubectl rollout restart deployment/<your-lychee-deployment>

Then check the rollout status:

kubectl rollout status deployment/<your-lychee-deployment>

You can also verify that the environment variables are available inside the running pod:

kubectl exec -it <pod-name> -- sh -c \
'printenv | grep -E "S3_ENABLED|AWS_ENDPOINT|AWS_DEFAULT_REGION|AWS_BUCKET|AWS_USE_PATH_STYLE_ENDPOINT"'

The result should contain values similar to:

S3_ENABLED=true
AWS_DEFAULT_REGION=fr-par
AWS_ENDPOINT=https://s3.fr-par.scw.cloud
AWS_BUCKET=my-lychee-photo-storage
AWS_USE_PATH_STYLE_ENDPOINT=false

If Lychee is using cached Laravel configuration, clear the application cache inside the pod:

kubectl exec -it <pod-name> -- php artisan optimize:clear

Then restart the pod or deployment again:

kubectl rollout restart deployment/<your-lychee-deployment>

The following exception:

RuntimeException: A facade root has not been set

may be a secondary exception generated while Laravel is handling an earlier bootstrap or configuration failure. Therefore, if correcting the endpoint does not solve the problem, please include the log lines immediately before that stack trace.

You can retrieve more complete logs with:

kubectl logs <pod-name> --previous

or:

kubectl logs <pod-name> --tail=200

In summary, I would try these changes first:

1. Change AWS_ENDPOINT to https://s3.fr-par.scw.cloud.
2. Quote all values in ConfigMap.data.
3. Move the access key and secret key into a Kubernetes Secret.
4. Clear Laravel's cached configuration.
5. Restart the deployment.
6. Inspect the log lines before the facade exception if the issue remains.

[FredPraca]

Thanks for your answer but alas, I put variables between double-quotes, updated endpoint, moved secrets into a secret and nothing changed.

About clearing the Laravel's cached config, the only volumes I mount do not seem to contain any configuration. Moreover, erasing the pod should not get any remaining configuration.

FYI, I use Podman and not k8s but I prefer using k8s deployment files instead of a podman compose file.

Here is the full log

5380ec6c3f71 🚀 Starting Lychee entrypoint...
5380ec6c3f71 🔍 Validating environment variables...
5380ec6c3f71 ✅ Valid database connection type: pgsql
5380ec6c3f71 ✅ Valid environment: production
5380ec6c3f71 ✅ Core variables validated
5380ec6c3f71 🎉 Environment validation complete!
5380ec6c3f71 ⏳ Waiting for database to be ready...
5380ec6c3f71 ✅ Database port is open!
5380ec6c3f71 Validating and setting PUID/PGID
5380ec6c3f71   User UID: 33
5380ec6c3f71   User GID: 33
5380ec6c3f71 🔍 Validating permissions...
5380ec6c3f71 ⏰ Set Permissions for Lychee folders...
5380ec6c3f71 ✅ Permissions set securely
5380ec6c3f71 ⏰ Set Permissions for Upload/dist folders (this may take a while)...
5380ec6c3f71 ✅ Permissions set securely
5380ec6c3f71 Checking RUN_AS_ROOT setting
5380ec6c3f71 ✅ Will drop privileges to www-data user
5380ec6c3f71 🧹 Clearing bootstrap cache...
5380ec6c3f71 🌐 Starting Lychee in web mode...
5380ec6c3f71 🔄 Running database migrations...
5380ec6c3f71 PHP Fatal error:  Uncaught RuntimeException: A facade root has not been set. in /app/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php:360
5380ec6c3f71 Stack trace:
5380ec6c3f71 #0 /app/app/Exceptions/Handler.php(435): Illuminate\Support\Facades\Facade::__callStatic()
5380ec6c3f71 #1 /app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(641): App\Exceptions\Handler->report()
5380ec6c3f71 #2 /app/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(200): Illuminate\Foundation\Console\Kernel->reportException()
5380ec6c3f71 #3 /app/artisan(35): Illuminate\Foundation\Console\Kernel->handle()
5380ec6c3f71 #4 {main}
5380ec6c3f71   thrown in /app/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php on line 360

But now that I copied the full trace, I think that may be a problem with database containing paths on file system.
Is there any differences between paths when using file system or S3 ? Do I have to update database in some way ?

[illhammf]

Thanks for the full log. Based on this trace, I think the S3 endpoint is probably not the main issue anymore.

The relevant part is this:

Running database migrations...
PHP Fatal error: Uncaught RuntimeException: A facade root has not been set.

This usually means Laravel failed very early during the Artisan/bootstrap phase. Then, while Laravel was trying to report the original exception, the exception handler itself tried to use a facade before the application container was fully initialized.

So A facade root has not been set is probably a secondary error, not necessarily the real root cause.

About the storage question:

Is there any differences between paths when using file system or S3? Do I have to update database in some way?

In general, switching from local filesystem storage to S3 should not require manually updating the database just to make migrations run.

Existing media records may contain paths or identifiers related to already-uploaded files, so old images may need migration/copying later if you are moving existing data from local storage to S3. But that should affect media access/display, not crash php artisan migrate during application startup.

So I would not start by changing the database.

Since you are using Podman, I would now try to run Artisan manually with verbose output inside the container:

podman exec -it <container-name> php artisan migrate --force -vvv

If the container exits too quickly, start a temporary shell with the same image and environment:

podman run --rm -it \
  --entrypoint sh \
  --env-file <your-env-file> \
  <lychee-image>

Then inside the container, run:

php artisan migrate --force -vvv

This should hopefully reveal the original exception before Laravel's exception handler triggers the facade error.

I would also verify the effective environment variables inside the running container:

printenv | grep -E "APP_|DB_|S3_|AWS_|FILESYSTEM|CACHE|QUEUE|SESSION"

In particular, I would check:

APP_KEY
APP_ENV
APP_URL
DB_CONNECTION
DB_HOST
DB_PORT
DB_DATABASE
DB_USERNAME
DB_PASSWORD
S3_ENABLED
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION
AWS_BUCKET
AWS_ENDPOINT
AWS_USE_PATH_STYLE_ENDPOINT

For Scaleway Object Storage in fr-par, I would keep the endpoint as:

AWS_ENDPOINT=https://s3.fr-par.scw.cloud
AWS_DEFAULT_REGION=fr-par
AWS_USE_PATH_STYLE_ENDPOINT=false

One more possible angle: if Lychee expects a specific filesystem variable, it may be worth checking whether the container expects S3_ENABLED only, or also something like FILESYSTEM_DISK, DEFAULT_FILESYSTEM_DISK, or another storage-related variable. A mismatch there could make Laravel build an invalid filesystem configuration during bootstrap.

Also, deleting the pod/container should normally remove runtime cache, unless cached config is baked into the image or generated again during startup. Your log shows:

Clearing bootstrap cache...

so the entrypoint does appear to be clearing Laravel bootstrap cache already.

At this point, I think the most useful next step is to get the output of:

php artisan migrate --force -vvv

from inside the container. That should expose the original error. The database-path/S3-path theory seems less likely, because the crash happens while running migrations/bootstrap, not while reading existing media files.