# Using S3 compatible Object Storage

Encore supports using S3-compatible object storage for both input and output files. S3 URLs should be specified in the format `s3://<BUCKET>/<KEY>`.

For details on submitting jobs with S3 inputs and outputs, see the [API and Job Submission](API-and-job-submission.md) guide and the [Input Configuration](Input-configuration.md) guide.

## Input from S3

When an S3 URL is used as input, encore will create a presigned URL that is then passed to ffmpeg for reading the file. Note that presigned URLs will be shown in the logs by default. If this is not desirable, you can configure encore to use a log configuration that masks the presign query parameters (see Configuration section).

## Output to S3

When an S3 URL is used for `outputFolder`, the transcoding output will first be stored locally in a temporary directory, and then uploaded to S3 once transcoding is finished.

## Authentication

AWS credentials are read using the AWS `DefaultCredentialsProvider`. This means credentials can be provided in multiple ways, including:
- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
- AWS credentials file (`~/.aws/credentials`)
- IAM role (when running on EC2, ECS, or other AWS services)
- EC2 instance metadata service

For more details, see the [AWS SDK documentation](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html).

S3 URLs can also be accessed in anonymous mode, which corresponds to using the `--no-sign-request` flag with the AWS CLI. When anonymous access is enabled, any configured S3 access key or secret key will be ignored and multipart upload will be disabled (as the S3 SDK does not support multipart upload with anonymous access).

## Multipart Upload

Encore uses the AWS CRT-based S3 client for efficient multipart uploads. The multipart upload behavior can be configured to optimize performance based on your network and file sizes. For detailed descriptions of these properties, see the [AWS SDK documentation](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3CrtAsyncClientBuilder.html).

## Configuration

S3 support must be explicitly enabled. All S3-related configuration properties are listed below:

| Property | Env Variable | Type | Default | Description |
| --- | --- | --- | --- | --- |
| `remote-files.s3.enabled` | `REMOTEFILES_S3_ENABLED` | Boolean | `false` | Enable S3 support |
| `remote-files.s3.anonymousAccess` | `REMOTEFILES_S3_ANONYMOUSACCESS` | Boolean | `false` | Enable anonymous access (no credentials required) |
| `remote-files.s3.endpoint` | `REMOTEFILES_S3_ENDPOINT` | String | - | Custom S3 endpoint for S3-compatible services (e.g., MinIO) |
| `remote-files.s3.usePathStyle` | `REMOTEFILES_S3_USEPATHSTYLE` | Boolean | `true` | Use path-style access (useful for S3-compatible services) |
| `remote-files.s3.presignDurationSeconds` | `REMOTEFILES_S3_PRESIGNDURATIONSECONDS` | Long | `43200` (12 hours) | Duration of presigned URLs for input files |
| `remote-files.s3.uploadTimeoutSeconds` | `REMOTEFILES_S3_UPLOADTIMEOUTSECONDS` | Long | `3600` (1 hour) | Timeout for upload operations |
| `remote-files.s3.multipart.minimumPartSize` | `REMOTEFILES_S3_MULTIPART_MINIMUMPARTSIZE` | Long (bytes) | Auto | Minimum size of each part in a multipart upload |
| `remote-files.s3.multipart.threshold` | `REMOTEFILES_S3_MULTIPART_THRESHOLD` | Long (bytes) | Auto | File size threshold for using multipart upload |
| `remote-files.s3.multipart.targetThroughputGbps` | `REMOTEFILES_S3_MULTIPART_TARGETTHROUGHPUTGBPS` | Double | Auto | Target throughput in Gbps |
| `remote-files.s3.multipart.maxConcurrency` | `REMOTEFILES_S3_MULTIPART_MAXCONCURRENCY` | Integer | Auto | Maximum number of concurrent upload operations |
| `logging.config` | `LOGGING_CONFIG` | String | - | Set to `classpath:logback-json-mask-s3-presign.xml` to mask presigned URLs in logs |

**Region:** The AWS region is determined from the `aws.region` system property or `AWS_REGION` environment variable (defaults to `us-east-1` if neither is set).

**Credentials:** See the Authentication section above for details on how credentials are provided.

Example YAML configuration:
```yaml
remote-files:
  s3:
    enabled: true
    presignDurationSeconds: 43200
    uploadTimeoutSeconds: 3600
    multipart:
      minimumPartSize: 8388608
      threshold: 8388608
      targetThroughputGbps: 10.0
      maxConcurrency: 50
```

## Example job with s3 input/output

```json
{
  "profile": "x264-animated",
  "outputFolder": "s3://my-output-bucket/transcoded/",
  "baseName": "my-video",
  "inputs": [
    {
      "uri": "s3://my-input-bucket/sources/input-video.mp4",
      "type": "AudioVideo"
    }
  ]
}
```

The transcoded files will be uploaded to `s3://my-output-bucket/transcoded/` with filenames based on the `baseName` and encoding profile settings.