Skip to content

Add S3VectorsCreateVectorBucketOperator#65968

Merged
kaxil merged 1 commit intoapache:mainfrom
john-jac:add-s3vectors-create-vector-bucket
Apr 28, 2026
Merged

Add S3VectorsCreateVectorBucketOperator#65968
kaxil merged 1 commit intoapache:mainfrom
john-jac:add-s3vectors-create-vector-bucket

Conversation

@john-jac
Copy link
Copy Markdown
Contributor

What

Add a new operator for Amazon S3 Vectors to create vector buckets.

S3 Vectors provides native vector storage in Amazon S3 for AI/ML applications, enabling similarity search queries at scale.

How

  • New S3VectorsCreateVectorBucketOperator using AwsBaseOperator[AwsBaseHook] pattern (no custom hook)
  • Supports encryption_configuration (SSE-S3/SSE-KMS) and tags parameters
  • Returns the vector bucket ARN

Files

  • operators/s3vectors.py — operator implementation
  • test_s3vectors.py — unit tests (3 tests)
  • example_s3vectors.py — system test
  • s3vectors/s3vectors.rst — docs with example
  • provider.yaml + get_provider_info.py — integration + operator entries
  • docs/operators/index.rst — toctree entry

Testing

  • Unit tests: 3 passed
  • mypy: no issues
  • Static checks (prek): 57 passed, 0 PR-related failures

@john-jac john-jac requested a review from o-nikolas as a code owner April 27, 2026 18:43
@john-jac john-jac force-pushed the add-s3vectors-create-vector-bucket branch from 50fa4a8 to 44e71f0 Compare April 27, 2026 20:04
Comment thread providers/amazon/src/airflow/providers/amazon/aws/operators/s3vectors.py Outdated
Comment thread providers/amazon/tests/system/amazon/aws/example_s3vectors.py Outdated
Comment thread providers/amazon/tests/unit/amazon/aws/operators/test_s3_vectors.py
Comment thread providers/amazon/docs/operators/s3vectors/s3vectors.rst
@john-jac john-jac force-pushed the add-s3vectors-create-vector-bucket branch from 44e71f0 to 52f0b3f Compare April 27, 2026 22:45
Comment thread providers/amazon/docs/operators/s3vectors/s3vectors.rst
Comment thread providers/amazon/tests/system/amazon/aws/example_s3vectors.py Outdated
@john-jac john-jac force-pushed the add-s3vectors-create-vector-bucket branch 2 times, most recently from cbc9c81 to 01bc7d0 Compare April 28, 2026 00:06
- New operator to create S3 Vectors vector buckets
- Idempotent: if_exists parameter (skip/fail) handles ConflictException
- Uses AwsBaseOperator[AwsBaseHook] with prune_dict
- Unit tests, system test with teardown, and docs included
- provider.yaml and get_provider_info.py updated
@john-jac john-jac force-pushed the add-s3vectors-create-vector-bucket branch from 01bc7d0 to 031259d Compare April 28, 2026 04:41
@kaxil kaxil merged commit f7f8ebf into apache:main Apr 28, 2026
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants