Repository navigation
✨feat(source-s3): adjust file record message protocol - #57498
Merged
Aldo Gonzalez (aldogonzalez8) merged 22 commits intoMay 5, 2025
Merged
Aldo Gonzalez (aldogonzalez8) merged 22 commits into
Aldo Gonzalez (aldogonzalez8) merged 22 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
…-message-protocol Merge from master
Contributor
Author
|
/approve-regression-tests
|
Contributor
Author
|
/approve-regression-tests
|
1 of 2 tasks
Contributor
Author
|
/approve-regression-tests
|
Baz (bazarnov)
approved these changes
Apr 23, 2025
| Returns: | ||
| str: The properly formatted S3 URI | ||
| """ | ||
| file_path = file.uri.split("#")[0] if isinstance(file, RemoteFileInsideArchive) else file.uri |
Contributor
There was a problem hiding this comment.
Would it be better to use the re and search for the certain pattern here?
|
|
||
| logger.debug(f"try to open {file.uri}") | ||
| try: | ||
| s3_uri = self._construct_s3_uri(file) |
Contributor
There was a problem hiding this comment.
+1 for using the construction here
Contributor
Author
|
/approve-regression-tests |
Contributor
Author
|
/approve-regression-tests
|
Contributor
Author
|
/approve-regression-tests
|
Contributor
Author
|
/approve-regression-tests
|
Contributor
Author
|
/approve-regression-tests
|
Aldo Gonzalez (aldogonzalez8)
deleted the
aldogonzalez8/source-s3/adjust-file-record-message-protocol
branch
May 5, 2025 16:00
letiescanciano
added a commit
that referenced
this pull request
May 6, 2025
…ectors-in-sidebar * master: (3385 commits) fix(Source-Harvest): missing schema for `time_team`, `users`, `time_entries`, `company`, & `project_assignments` streams. (#59215) chore: add `poe_tasks.toml` for all java connectors (#59670) ci: add Dockerfile base image definitions, along with sample `.sh` build scripts (#59212) Correct Dest SF readme. (#59658) ✨ TikTok Marketing Source: By country daily/hourly report streams (#55681) source-vitally contribution from aidanlister (#58062) Upgrade Gradle to 8.14. (#59662) source-awin-advertiser contribution from ryanmcg2203 (#58632) 🐛 Source asana: add new stream portfolio_items (#59224) docs: add pre 1.0 semver exception to breaking changes ✨ Source Brevo: fix pagination on contact stream (#59652) Load CDK: Pre-Speed Cleanup: Remove CheckpointManager interface/abstract class (#59218) Load CDK: Pre-Speed Cleanup: Remove Sync/Stream manager interfaces (#59217) Load CDK: Non-functional Cleanup: Remove Launcher Interface/Secondary (#59213) fix(live-tests): use first 8 symbols of connection id instead of hashing (#59653) ✨feat(source-s3): adjust file record message protocol (#57498) feat(source-zendesk-support): new article attachments with file transfer support (#56419) source-mixmax contribution from cbuckle1 (#59160) ✨feat(source-google-drive): adjust file record message protocol (#57496) Typo fix (#59209) ...
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This PR updates the
source-s3connector to comply with the latest file transfer record protocol.Resolves: https://github.com/airbytehq/airbyte-internal-issues/issues/12341
How
source-s3connector's implementation to align with the latest protocol specifications.Review guide
1.
airbyte-integrations/connectors/source-s3/source_s3/v4/stream_reader.py: returns file reference and file metadata.User Impact
Can this PR be safely reverted and rolled back?