Documentation Improvement - Connecting self-hosted Storage API to S3

[mohannadhussain]

I posted about this in Discord and wanted to share my findings here in hopes of improving documentation for others going forward.

Connecting Supabase Storage API to AWS S3

The instructions here apply to the self-hosted version of Supabase, i.e. the one used via docker-compose

Step 1: What you need

1. An AWS Access key ID and secret access key.
2. An existing S3 bucket that is accessible to your access key.

Step 2: Configuration

In your supabase directory, open docker-compose.yml in your favorite editor. Locate the storage section and Change the following environment varialbes:

• STORAGE_BACKEND: file to STORAGE_BACKEND: s3
• REGION to your AWS Region, e.g. us-east-1
• GLOBAL_S3_BUCKET to the name of your AWS S3 bucket
• (Optional) TENANT_ID is the top-level directory (i.e. s3 path prefix), you can change it if you want.

Now, you need to setup AWS authentication, which can be done in one of two ways, depending on your preferences:

1. In the same block in docker-compose.yml, you can add environment variables for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, or
2. Create a local files called credentials then map it in docker-compose.yml under volumes like so: ./credentials:/root/.aws/credentials - the file's contents would look like this:

[default]
aws_access_key_id = XXX
aws_secret_access_key = XXX

Step 3: Test it

Boot up your setup with docker-compose up, use the UI to create a new bucket and upload a file into it. Now, navigate to your S3 bucket in the AWS console and verify your file was uploaded there. The path looks like this {TENANT_ID from docker-compose.yml}/{your bucket name}/{your file name}

A few gotchas to keep in mind:

1. Any pre-existing buckets you've previously created on the local file system will continue to appear in the UI - don't be alarmed.
2. When you create a new bucket without uploading a file, you will not see a folder created for that bucket into S3. It seems to only create the folder corresponding to the bucket when a file is actually uploaded.

[Conobi]

Hey!
Thanks for your help, but I still understand how to configure properly S3 for Supabase, especially for a Wasabi setup.
Which environment variable must be set to what? What is the TENANT_ID?
In a setup where the service URL is s3.eu-central-1.wasabisys.com and the bucket's name foobar, what environment variable would be set?

[mohannadhussain]

I don't know anything about Wasabi, but the environment variables you need are spelled out in step 2 above, ie: STORAGE_BACKEND, REGION, GLOBAL_S3_BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.

TENANT_ID is optional. If you don't specify it, it default to "stub" - which is going to be the name of the root folder in S3 where Supa uploads things. Once you get it working, it will make a lot more sense.

[stoickeyboard]

@Donokami Did you ever figure it out? I'm trying to connect supabase to Backblaze without much luck. But I do believe you have to set the GLOBAL_S3_ENDPOINT environment variable. In your case you'd set it to GLOBAL_S3_ENDPOINT=https://s3.eu-central-1.wasabisys.com and for the others REGION=eu-central-1, GLOBAL_S3_BUCKET=foobar, AWS_ACCESS_KEY_ID=<key>, and AWS_SECRET_ACCESS_KEY=<secret_access_key> all inside of .env

Then inside of docker-compose.yml under storage: ->environment: you'll have
GLOBAL_S3_BUCKET: ${GLOBAL_S3_BUCKET} etc.

@mohannadhussain Thanks for writing this all up. Super helpful even though I couldn't get it working with Backblaze. It's kinda interesting that these environment variables were left out completely. You'd at least think that they'd be commented out as some of the others were. I've spent hours trying to figure this out. It's surprising that there's not more comments about it

[stoickeyboard]

@Donokami There was something wrong with my nginix setup. What I responded with actually did work for me.

Thanks again @mohannadhussain

[mohannadhussain]

Glad my tiny contribution became handy for others. And thank you for your additions :)

[Anj-interactivated]

@Donokami have you managed to make it work. Same challenge here ... I think the config is fine but getting errors

Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.: TemporaryRedirect

httpStatusCode: 307

edited: solved by including region in the GLOBAL_S3_ENDPOINT: https://s3.eu-west-1.wasabisys.com, still requires REGION: eu-west-1 to be set as well

[y4my4my4m]

@mohannadhussain amazing, thank you for sharing this!