Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolving EMR deprecated warnings #39743

Merged
merged 9 commits into from
May 23, 2024
Merged

Conversation

dirrao
Copy link
Contributor

@dirrao dirrao commented May 21, 2024

Resolving EMR deprecated warnings

related: #39485

@vincbeck
Copy link
Contributor

Static checks are failing

@ferruzzi
Copy link
Contributor

I'm not sure what this is meant to be fixing, but we definitely don't want to be adding waiter_delay and waiter_max_attempts to every operator in every system test...

Copy link
Contributor

@o-nikolas o-nikolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also agree from the description and code I have no idea what issue is attempting to be fixed here. Can you provide more details to make your PR easier for others to review?

@dirrao
Copy link
Contributor Author

dirrao commented May 22, 2024

@o-nikolas and @ferruzzi
We have marked some of the fields as deprecated. To resolve warnings we have made these code changes.

As per the recommendation from @Taragolis, Fix the operator code or adjust the operator default values at the time of removing deprecated code.

[tests/system/providers/amazon/aws/example_emr.py](https://github.com/apache/airflow/tree/main/tests/system/providers/amazon/aws/example_emr.py)
tests/system/providers/amazon/aws/example_emr.py:143:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.
 [tests/system/providers/amazon/aws/example_emr_notebook_execution.py](https://github.com/apache/airflow/tree/main/tests/system/providers/amazon/aws/example_emr_notebook_execution.py)
tests/system/providers/amazon/aws/example_emr_notebook_execution.py:54:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:54:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_countdown has been deprecated to standardize naming conventions.  Please use waiter_max_attempts instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:75:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:75:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_countdown has been deprecated to standardize naming conventions.  Please use waiter_max_attempts instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:87:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:87:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_countdown has been deprecated to standardize naming conventions.  Please use waiter_max_attempts instead.  In the future this will default to None and defer to the waiter's default value.

airflow/providers/amazon/aws/operators/emr.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/operators/emr.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/operators/emr.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/operators/emr.py Outdated Show resolved Hide resolved
airflow/providers/amazon/aws/operators/emr.py Outdated Show resolved Hide resolved
dirrao and others added 5 commits May 22, 2024 21:01
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Copy link
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, fixing the operator(s) makes much more sense than changing the tests.

@dirrao dirrao requested a review from Taragolis May 23, 2024 03:56
@vincbeck vincbeck merged commit e565cea into apache:main May 23, 2024
42 checks passed
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants