Skip to content

(feat) Implement SourceUnitEnumChunker for S3 source #4270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

(feat) Implement SourceUnitEnumChunker for S3 source #4270

wants to merge 2 commits into from

Conversation

abmussani
Copy link
Contributor

Description:

This PR implements SourceUnitEnumChunker for S3 source. Each Object in the bucket will be considered as Unit.

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

@abmussani
implemented Source unit for S3.
d49cc3a 
Implemented. integration test
@abmussani abmussani requested review from a team as code owners June 26, 2025 16:54
@abmussani abmussani self-assigned this Jun 26, 2025
rosecodym
rosecodym requested changes Jun 26, 2025
View reviewed changes
Copy link
Collaborator

@rosecodym rosecodym left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requesting changes until we have an internal discussion about unit granularity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rosecodym rosecodym rosecodym requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@abmussani abmussani

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@abmussani @rosecodym