Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fail
LocalFilesystemToGCSOperator
if src does not exist (#22772)
Fix #22705. Fail LocalFilesystemToGCSOperator if the src file does not exist `src` argument of LocalFilesystemToGCSOperator accept either list of source file path or a single source file path as a string. In the case of a single source file path we are using [glob](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/local_to_gcs.py#L111) to parse the file path and glob return empty list if file path does not exist. In the next step, we iterate on this list and call [hook api](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/local_to_gcs.py#L123) to update the file since the list is empty control is not going inside loop and task is succeeding even if the source file is not available. Change - Raise an exception if [filepath](https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/local_to_gcs.py#L111) list is empty After this change below task will fail if example-text.txt does not exist ``` upload_file = LocalFilesystemToGCSOperator( task_id="upload_file", src="example-text.txt", dst=DESTINATION_FILE_LOCATION, bucket=BUCKET_NAME, ) ```
- Loading branch information