Fix GCSToGCSOperator ignoring replace=False for single-file copy#66014
Merged
shahar1 merged 1 commit intoapache:mainfrom May 5, 2026
Merged
Fix GCSToGCSOperator ignoring replace=False for single-file copy#66014shahar1 merged 1 commit intoapache:mainfrom
shahar1 merged 1 commit intoapache:mainfrom
Conversation
When source_object is a single object (no wildcard) and the destination already exists, _copy_source_without_wildcard fell into a fallback branch that called _copy_single_object unconditionally, ignoring replace=False. Closes apache#66009
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
Contributor
|
cc @VladaZakharova for review |
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
79 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Respect
replace=Falsein the single-file fallback path ofGCSToGCSOperator._copy_source_without_wildcard. When the source is asingle object (no wildcard),
_ignore_existing_filescorrectly emptiesobjectsif the destination already has the file, but the subsequentif len(objects) == 0 and prefix:branch then called_copy_single_objectunconditionally, overwriting the destination.This adds a destination existence check guarded by
not self.replaceinside that branch, skipping the copy when the destination already exists.
Alternative considered
I also considered limiting the fallback itself to the case where the
source list was empty before
_ignore_existing_filesran. The chosenapproach is smaller and keeps the intent local to where the copy happens;
happy to switch to the alternative if reviewers prefer it.
closes: #66009
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.7) following the guidelines