Skip to content

Add S3ArtifactService for Amazon S3 artifact storage support #3872

@miyannishar

Description

@miyannishar

** Please make sure you read the contribution guide and file the issues in the right place. **

Contribution guide.

Is your feature request related to a problem? Please describe.

ADK currently provides GcsArtifactService for Google Cloud Storage and FileArtifactService for local development, but there is no native artifact service for Amazon S3. This creates friction for developers and organizations using AWS infrastructure, as they must either:

  • Use FileArtifactService (not production-ready, limited scalability)
  • Use GcsArtifactService (requires cross-cloud setup, additional complexity)
  • Implement their own S3 service from scratch (time-consuming, error-prone)

For AWS-native deployments (EC2, ECS, Lambda, EKS), developers need a first-class S3 artifact storage solution that integrates seamlessly with their existing infrastructure and IAM policies.

Describe the solution you'd like

Implement S3ArtifactService as a production-ready artifact storage backend with the following characteristics:

Core Features:

  • Extends BaseArtifactService following the same patterns as GcsArtifactService
  • Supports session-scoped artifacts: {app}/{user}/{session}/{filename}/{version}
  • Supports user-scoped artifacts: {app}/{user}/user/{filename}/{version} (using user: prefix)
  • Automatic version management (0, 1, 2, ...)
  • Custom metadata support via S3 object metadata
  • URL encoding for special characters in filenames
  • Async/sync pattern using asyncio.to_thread

Technical Implementation:

  • Uses boto3 for S3 interactions
  • Added as optional dependency in pyproject.toml under extensions
  • No breaking changes to existing code
  • Full feature parity with GcsArtifactService

Developer Experience:

from google.adk.artifacts import S3ArtifactService

artifact_service = S3ArtifactService(
    bucket_name="my-adk-artifacts",
    region_name="us-east-1"
)

runner = Runner(agent=my_agent, artifact_service=artifact_service)

Describe alternatives you've considered

  1. Using GcsArtifactService with cross-cloud setup

    • Requires managing credentials across cloud providers
    • Additional latency for cross-cloud data transfer
    • Increased costs for egress between clouds
    • More complex IAM/permission management
  2. Using FileArtifactService for production

    • Not designed for production (single machine, no replication)
    • Limited by disk space
    • No durability guarantees (99.999999999% for S3 vs. disk failure)
    • Requires persistent volumes in containerized environments
  3. Implementing custom S3 wrapper

    • Every team duplicates effort
    • No standardization across projects
    • Potential bugs and inconsistencies
    • Maintenance burden on each team
  4. Using external artifact storage services

    • Additional service dependency
    • Extra cost
    • Learning curve for another tool
    • Less control over infrastructure

Metadata

Metadata

Assignees

No one assigned

    Labels

    services[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions