Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .env.acceptance.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#######################################
# Acceptance Runner
#######################################
ACCEPTANCE_TARGET=local
ACCEPTANCE_PROFILE=smoke
ACCEPTANCE_BASE_URL=http://127.0.0.1:5000
ACCEPTANCE_S3_ENDPOINT=http://127.0.0.1:5000/s3
ACCEPTANCE_TUS_ENDPOINT=http://127.0.0.1:5000/upload/resumable
ACCEPTANCE_REGION=us-east-1
ACCEPTANCE_RESOURCE_PREFIX=acc-local
# Matches the default dummy-data tenant; change this for non-default seeds or remote targets.
ACCEPTANCE_TENANT_ID=bjhaohmqunupljrqypxz
ACCEPTANCE_ALLOW_DESTRUCTIVE=false
ACCEPTANCE_ENABLE_ADMIN=false
ACCEPTANCE_ENABLE_CDN=false
ACCEPTANCE_ENABLE_RENDER=false
ACCEPTANCE_ENABLE_RLS_SETUP=false
ACCEPTANCE_ENABLE_VECTOR=false
ACCEPTANCE_ENABLE_ICEBERG=false
ACCEPTANCE_ENABLE_PATH_EDGES=false
ACCEPTANCE_ENABLE_WIRE=false
ACCEPTANCE_ENABLE_TUS=true
ACCEPTANCE_S3_FORCE_PATH_STYLE=true

#######################################
# Acceptance Credentials
#######################################
ACCEPTANCE_ADMIN_API_KEY=apikey
ACCEPTANCE_AUTHENTICATED_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJhdXRoZW50aWNhdGVkIiwic3ViIjoiMzE3ZWFkY2UtNjMxYS00NDI5LWEwYmItZjE5YTdhNTE3YjRhIiwiZW1haWwiOiJpbmlhbit0ZXN0MUBzdXBhYmFzZS5pbyIsImV4cCI6MTkzOTEwNzk4NSwiYXBwX21ldGFkYXRhIjp7InByb3ZpZGVyIjoiZW1haWwifSwidXNlcl9tZXRhZGF0YSI6e30sInJvbGUiOiJhdXRoZW50aWNhdGVkIn0.E-x3oYcHIjFCdUO1M3wKDl1Ln32mik0xdHT2PjrvN70
ACCEPTANCE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiYW5vbiIsImlhdCI6MTYxMzUzMTk4NSwiZXhwIjoxOTI5MTA3OTg1fQ.mqfi__KnQB4v6PkIjkhzfwWrYyF94MEbSC6LnuvVniE
ACCEPTANCE_SERVICE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoic2VydmljZV9yb2xlIiwiaWF0IjoxNjEzNTMxOTg1LCJleHAiOjE5MjkxMDc5ODV9.th84OKK0Iz8QchDyXZRrojmKSEZ-OuitQm_5DvLiSIc

#######################################
# S3 Client Credentials
#######################################
ACCEPTANCE_S3_ACCESS_KEY_ID=b585f311d839730f8a980a3457be2787
ACCEPTANCE_S3_SECRET_ACCESS_KEY=67d161a7a8a46a24a17a75b26e7724f11d56b8d49a119227c66b13b6595601fb
5 changes: 5 additions & 0 deletions .env.test.sample
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ REQUEST_X_FORWARDED_HOST_REGEXP=
VECTOR_ENABLED=true
VECTOR_S3_BUCKETS=supa-test-local-dev
ICEBERG_ENABLED=true
ICEBERG_WAREHOUSE=.
ICEBERG_CATALOG_URL=http://127.0.0.1:8181/v1
ICEBERG_CATALOG_AUTH_TYPE=token
ICEBERG_CATALOG_AUTH_TOKEN=token
ICEBERG_S3_DELETE_ENABLED=true
ICEBERG_BUCKET_DETECTION_MODE="BUCKET"

OTEL_METRICS_ENABLED=false
Expand Down
146 changes: 146 additions & 0 deletions .github/workflows/acceptance.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
name: Acceptance

on:
pull_request:
push:
branches:
- master
workflow_dispatch:
inputs:
profile:
description: Acceptance profile to run
required: true
default: smoke
type: choice
options:
- smoke
- core
- full
- wire
acceptance_environment:
description: Acceptance target to run
required: true
default: local
type: choice
options:
- local
- acceptance-remote
allow_destructive:
description: Allow destructive tests against remote targets
required: true
default: false
type: boolean

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}-${{ inputs.acceptance_environment || 'local' }}
cancel-in-progress: true

Comment thread
ferhatelmas marked this conversation as resolved.
jobs:
acceptance_local:
name: Local
if: ${{ github.event_name != 'workflow_dispatch' || inputs.acceptance_environment == 'local' }}
runs-on: blacksmith-4vcpu-ubuntu-2404
Comment thread
ferhatelmas marked this conversation as resolved.
timeout-minutes: 35
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Set up Node and npm
uses: ./.github/actions/setup-node-npm
with:
node-version: "24"
- name: Install dependencies
run: npm ci
- name: Prepare env files
run: |
cp .env.sample .env
cp .env.test.sample .env.test
cp .env.acceptance.sample .env.acceptance
- name: Typecheck acceptance suite
run: npm run acceptance:typecheck
- name: Run local acceptance profile
env:
ACCEPTANCE_PROFILE: ${{ inputs.profile || 'smoke' }}
run: |
mkdir -p data coverage/acceptance
chmod -R 777 data
npm run acceptance -- --profile "${ACCEPTANCE_PROFILE}"
- name: Upload acceptance artifacts
if: ${{ always() }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: acceptance-local
path: coverage/acceptance
if-no-files-found: ignore

acceptance_remote:
name: Remote
if: ${{ github.event_name == 'workflow_dispatch' && inputs.acceptance_environment != 'local' }}
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 20
environment: ${{ inputs.acceptance_environment }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
- name: Set up Node and npm
uses: ./.github/actions/setup-node-npm
with:
node-version: "24"
- name: Install dependencies
run: npm ci
- name: Typecheck acceptance suite
run: npm run acceptance:typecheck
- name: Run acceptance profile
env:
ACCEPTANCE_ALLOW_DESTRUCTIVE: ${{ inputs.allow_destructive }}
ACCEPTANCE_BASE_URL: ${{ vars.ACCEPTANCE_BASE_URL }}
ACCEPTANCE_ENABLE_ADMIN: ${{ vars.ACCEPTANCE_ENABLE_ADMIN }}
ACCEPTANCE_ENABLE_CDN: ${{ vars.ACCEPTANCE_ENABLE_CDN }}
ACCEPTANCE_ENABLE_ICEBERG: ${{ vars.ACCEPTANCE_ENABLE_ICEBERG }}
ACCEPTANCE_ENABLE_PATH_EDGES: ${{ vars.ACCEPTANCE_ENABLE_PATH_EDGES }}
ACCEPTANCE_ENABLE_RENDER: ${{ vars.ACCEPTANCE_ENABLE_RENDER }}
ACCEPTANCE_ENABLE_RLS_SETUP: ${{ vars.ACCEPTANCE_ENABLE_RLS_SETUP }}
ACCEPTANCE_ENABLE_TUS: ${{ vars.ACCEPTANCE_ENABLE_TUS }}
ACCEPTANCE_ENABLE_VECTOR: ${{ vars.ACCEPTANCE_ENABLE_VECTOR }}
ACCEPTANCE_ENABLE_WIRE: ${{ vars.ACCEPTANCE_ENABLE_WIRE }}
ACCEPTANCE_PROFILE: ${{ inputs.profile }}
ACCEPTANCE_REGION: ${{ vars.ACCEPTANCE_REGION || secrets.ACCEPTANCE_REGION }}
ACCEPTANCE_RESOURCE_PREFIX: ${{ vars.ACCEPTANCE_RESOURCE_PREFIX || secrets.ACCEPTANCE_RESOURCE_PREFIX }}
ACCEPTANCE_RLS_BUCKET: ${{ vars.ACCEPTANCE_RLS_BUCKET }}
ACCEPTANCE_RLS_READ_OBJECT: ${{ vars.ACCEPTANCE_RLS_READ_OBJECT }}
ACCEPTANCE_RLS_WRITE_PREFIX: ${{ vars.ACCEPTANCE_RLS_WRITE_PREFIX }}
ACCEPTANCE_S3_ENDPOINT: ${{ vars.ACCEPTANCE_S3_ENDPOINT }}
ACCEPTANCE_S3_FORCE_PATH_STYLE: ${{ vars.ACCEPTANCE_S3_FORCE_PATH_STYLE }}
ACCEPTANCE_TLS_REJECT_UNAUTHORIZED: ${{ vars.ACCEPTANCE_TLS_REJECT_UNAUTHORIZED }}
ACCEPTANCE_TENANT_ID: ${{ vars.ACCEPTANCE_TENANT_ID || secrets.ACCEPTANCE_TENANT_ID }}
ACCEPTANCE_TUS_ENDPOINT: ${{ vars.ACCEPTANCE_TUS_ENDPOINT }}
ACCEPTANCE_ADMIN_URL: ${{ vars.ACCEPTANCE_ADMIN_URL || secrets.ACCEPTANCE_ADMIN_URL }}
ACCEPTANCE_ADMIN_API_KEY: ${{ secrets.ACCEPTANCE_ADMIN_API_KEY }}
ACCEPTANCE_ANON_KEY: ${{ secrets.ACCEPTANCE_ANON_KEY }}
ACCEPTANCE_AUTHENTICATED_KEY: ${{ secrets.ACCEPTANCE_AUTHENTICATED_KEY }}
ACCEPTANCE_S3_ACCESS_KEY_ID: ${{ secrets.ACCEPTANCE_S3_ACCESS_KEY_ID }}
ACCEPTANCE_S3_SECRET_ACCESS_KEY: ${{ secrets.ACCEPTANCE_S3_SECRET_ACCESS_KEY }}
ACCEPTANCE_SERVICE_KEY: ${{ secrets.ACCEPTANCE_SERVICE_KEY }}
ACCEPTANCE_TARGET: remote
run: |
mkdir -p coverage/acceptance
npm run acceptance:run
- name: Upload acceptance artifacts
if: ${{ always() }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: acceptance-remote
path: coverage/acceptance
if-no-files-found: ignore
50 changes: 50 additions & 0 deletions acceptance/API_COVERAGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Acceptance API Coverage

The acceptance suite is black-box by design. Tests only use public HTTP, S3, TUS, and admin
surfaces so the same contracts can be run against the current TypeScript service or a future
Go/Rust rewrite.

## Default PR Coverage

These run in `smoke` / `core` profiles and are suitable for pull requests against local CI:

| Area | Covered APIs / behavior |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| Health | `/status`, `/version` |
| REST buckets | create, get, list/search, update, empty, delete |
| REST objects | upload, update, authenticated read/head/info, public read/head/info, delete |
| REST object operations | list-v1, list-v2, signed URL, batch signed URLs, signed upload URL, copy, move, bulk delete |
| S3 buckets | CreateBucket, HeadBucket, ListBuckets, GetBucketLocation, GetBucketVersioning, DeleteBucket |
| S3 objects | PutObject, HeadObject, GetObject, Range GetObject, CopyObject, DeleteObject, DeleteObjects |
| S3 listing | ListObjectsV2, ListObjects V1 with delimiter/common prefixes |
| S3 multipart | CreateMultipartUpload, UploadPart, ListParts, CompleteMultipartUpload, UploadPartCopy, AbortMultipartUpload, ListMultipartUploads |
| TUS | OPTIONS, POST create, HEAD offset, PATCH resume, DELETE termination, full upload through `tus-js-client`, signed TUS upload |

Comment thread
ferhatelmas marked this conversation as resolved.
## Wire Profile Coverage

These run in the `wire` profile in addition to smoke coverage:

| Area | Covered APIs / behavior |
| ---------- | ----------------------------------------------------------------------- |
| Wire/SigV4 | raw `aws-chunked` PutObject and UploadPart, trailer-signature rejection |

## Opt-In Coverage

These require target-specific capabilities and are off by default:

| Capability | Enable with | Covered APIs / behavior |
| ---------- | --------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Admin | `ACCEPTANCE_ENABLE_ADMIN=true` plus admin URL/API key | admin status, API key protection, tenant reads, tenant migration state, metrics config, queue/migration validation, JWKS validation/status, orphan scan validation, S3 credential create/list/delete |
| CDN | `ACCEPTANCE_ENABLE_CDN=true` | `/cdn/:bucket/*` cache purge |
| Render | `ACCEPTANCE_ENABLE_RENDER=true` | public, authenticated, and signed image transformation routes |
| RLS | `ACCEPTANCE_ENABLE_RLS_SETUP=true` plus anon/authenticated keys and RLS resource config | authenticated allow and anon deny for read/write on configured policies |
| Path edges | `ACCEPTANCE_ENABLE_PATH_EDGES=true` | list-v2 preservation for object names with empty path segments, only on targets whose blob backend accepts those names |
| Vector | `ACCEPTANCE_ENABLE_VECTOR=true` | vector bucket, index, put/get/list/query/delete lifecycle |
| Iceberg | `ACCEPTANCE_ENABLE_ICEBERG=true` | analytics bucket, catalog config, namespace, table create/list/load/head/drop |

## Intentionally Gated

Remote runs should set `ACCEPTANCE_TARGET=remote`. Destructive tests are blocked on remote targets
unless `ACCEPTANCE_ALLOW_DESTRUCTIVE=true` is set. Admin, CDN, render, vector, Iceberg, and RLS
tests are capability-gated because they depend on tenant features or credentials that are not
universally available.
101 changes: 101 additions & 0 deletions acceptance/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Acceptance Tests

This suite verifies Supabase Storage as a black-box service. It is intentionally separate from
`src/test`: acceptance tests must talk to a running target through HTTP, S3, TUS, or admin
endpoints only. Do not import `src/app`, use `app.inject`, or call storage/database classes here.
See [`API_COVERAGE.md`](./API_COVERAGE.md) for the API/feature coverage inventory.

## Profiles

- `smoke` - fast target sanity plus REST and S3 lifecycle checks.
- `core` - smoke plus broader protocol behavior such as TUS and multipart.
- `wire` - smoke plus raw HTTP/SigV4 cases such as `aws-chunked` bodies and trailer negatives.
- `full` - all acceptance tests allowed by the configured target and capability gates.

## Run Against An Existing Target

The runner loads `.env.acceptance`. To use a different acceptance env file, set
`ACCEPTANCE_ENV_FILE`.

```bash
ACCEPTANCE_BASE_URL=http://127.0.0.1:5000 \
ACCEPTANCE_S3_ENDPOINT=http://127.0.0.1:5000/s3 \
ACCEPTANCE_SERVICE_KEY="..." \
ACCEPTANCE_S3_ACCESS_KEY_ID="..." \
ACCEPTANCE_S3_SECRET_ACCESS_KEY="..." \
npm run acceptance:run -- --profile smoke
```

For a hosted target with a path prefix, set `ACCEPTANCE_BASE_URL` to the public storage base,
for example `https://project.supabase.co/storage/v1`. Relative URLs returned by the API are
resolved against that base so prefix behavior is covered. Set `ACCEPTANCE_TARGET=remote` for
hosted targets; destructive tests then require `ACCEPTANCE_ALLOW_DESTRUCTIVE=true`.

For local targets on a non-default port, set `ACCEPTANCE_BASE_URL` explicitly.

## Managed Local Run

```bash
cp .env.sample .env
cp .env.test.sample .env.test
cp .env.acceptance.sample .env.acceptance
npm run acceptance -- --profile smoke
```

This restarts local infra, seeds dummy data, starts the TypeScript server from `.env.test` plus
`.env`, waits for `/status`, runs the acceptance profile, and then stops the server. Set
`ACCEPTANCE_SKIP_INFRA=true` to reuse already-running local infra.

For local backend variants, put server/runtime changes in `.env` or `.env.test`. Keep
`.env.acceptance` limited to acceptance runner inputs such as target URLs, client credentials,
capability gates, and resource naming.

## GitHub Environments

The workflow dispatch `acceptance_environment` input uses `local` for the managed local run. Any
other option is treated as a GitHub Environment name and is used to populate `.env.acceptance` at
runtime. Store non-secret values such as `ACCEPTANCE_BASE_URL`, `ACCEPTANCE_REGION`, and capability
flags as environment variables, and store credentials such as `ACCEPTANCE_SERVICE_KEY` and S3
secrets as environment secrets.

## Useful Configuration

| Variable | Meaning |
| --------------------------------- | ---------------------------------------------------------------------------- |
| `ACCEPTANCE_BASE_URL` | REST base URL. Defaults to `http://127.0.0.1:5000`. |
| `ACCEPTANCE_S3_ENDPOINT` | S3 endpoint. Defaults to `$ACCEPTANCE_BASE_URL/s3`. |
| `ACCEPTANCE_TUS_ENDPOINT` | TUS endpoint. Defaults to `$ACCEPTANCE_BASE_URL/upload/resumable`. |
| `ACCEPTANCE_ADMIN_URL` | Admin API base URL for admin tests. |
| `ACCEPTANCE_SERVICE_KEY` | Service role JWT for REST tests. |
| `ACCEPTANCE_S3_ACCESS_KEY_ID` | S3 protocol access key. |
| `ACCEPTANCE_S3_SECRET_ACCESS_KEY` | S3 protocol secret. |
| `ACCEPTANCE_REGION` | S3 signing region. Defaults to `us-east-1`. |
| `ACCEPTANCE_RESOURCE_PREFIX` | Prefix for all resources created by this run. |
| `ACCEPTANCE_ENABLE_ADMIN` | Enables admin route tests. Requires admin URL and API key. |
| `ACCEPTANCE_ENABLE_CDN` | Enables CDN purge tests. Requires purge-cache support on the target tenant. |
| `ACCEPTANCE_ENABLE_RENDER` | Enables image transformation tests. |
| `ACCEPTANCE_ENABLE_RLS_SETUP` | Enables RLS tests; requires service, anon, authenticated keys and resources. |
| `ACCEPTANCE_ENABLE_VECTOR` | Enables vector bucket API tests. |
| `ACCEPTANCE_ENABLE_ICEBERG` | Enables Iceberg catalog API tests. |
| `ACCEPTANCE_ENABLE_PATH_EDGES` | Enables object-name edge tests that require a backend accepting `//` names. |
| `ACCEPTANCE_ENABLE_WIRE` | Enables wire-level tests outside the `wire` / `full` profiles. |
| `ACCEPTANCE_RLS_BUCKET` | Bucket used by opt-in RLS tests. Defaults to local dummy `bucket2`. |
| `ACCEPTANCE_RLS_READ_OBJECT` | Existing object readable by authenticated role and denied to anon. |
| `ACCEPTANCE_RLS_WRITE_PREFIX` | Prefix where authenticated role may upload and anon may not. |
| `ACCEPTANCE_ALLOW_DESTRUCTIVE` | Required for destructive tests when `ACCEPTANCE_TARGET=remote`. |

Comment thread
ferhatelmas marked this conversation as resolved.
## HTTPS And Wire Tests

The `wire` profile includes smoke coverage and uses a raw SigV4 client for `aws-chunked`
payloads. To verify proxy/TLS behavior, point `ACCEPTANCE_S3_ENDPOINT` at an HTTPS URL, for
example:

```bash
ACCEPTANCE_S3_ENDPOINT=https://storage.localhost/s3 \
ACCEPTANCE_TLS_REJECT_UNAUTHORIZED=false \
npm run acceptance:run -- --profile wire
```

`ACCEPTANCE_TLS_REJECT_UNAUTHORIZED=false` sets `NODE_TLS_REJECT_UNAUTHORIZED=0` in the runner
for local self-signed certificates. Do not use it for remote runs unless the target is explicitly
provisioned for that.
Loading
Loading