Update redshift_to_s3_operator.py#10336
Update redshift_to_s3_operator.py#10336selvamshan wants to merge 4 commits intoapache:v1-10-stablefrom
Conversation
…che#7276) If a task is skipped by BranchPythonOperator, BaseBranchOperator or ShortCircuitOperator and the user then clears the skipped task later, it'll execute. This is probably not the right behaviour. This commit changes that so it will be skipped again. This can be ignored by running the task again with "Ignore Task Deps" override. (cherry picked from commit 1cdab56)
(cherry-picked from commit 4aa05a7)
S3ToRedshiftTransfer and RedshiftToS3Transfer with temporary token credentials apache#9970 Aws ref: https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-authorization.html example copy customer from 's3://mybucket/mydata' credentials 'aws_iam_role=arn:aws:iam::<account-id>:role/<role-name>;master_symmetric_key=<master-key>'
|
Hello @selvamshan -> I think that change is already quite stala and we are already moving away from making any fixes to 1.10 operators - instead - we'd rather implement them in master and release backport providers. I am planning to release the 2nd wave of backport providers this week after 1.10.12 gets released (#10014 ) - so if you would like to get that in, I'd love if you make a PR directly to master. I thin the change is rather small. Please ping me on github if you create one and I will fast-track review it if you would like to make it to the 2nd wave of backports. The operator moved in Airflow 2.0 (and in backport packages) - and you can install it by installing Shoudl we close this ticket? |
4 participants
S3ToRedshiftTransfer and RedshiftToS3Transfer with temporary token credentials #9970
Aws ref: https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-authorization.html
example
copy customer from 's3://mybucket/mydata'
credentials
'aws_iam_role=arn:aws:iam:::role/;master_symmetric_key='
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.