Skip to content

Fix flaky Glue OpenLineage test that calls the real AWS endpoint - #70500

Merged
vincbeck merged 1 commit into
apache:mainfrom
FrankYang0529:airflow-flaky-test_inject_parent_job_info_with_resume_on_retry
Jul 27, 2026
Merged

Fix flaky Glue OpenLineage test that calls the real AWS endpoint#70500
vincbeck merged 1 commit into
apache:mainfrom
FrankYang0529:airflow-flaky-test_inject_parent_job_info_with_resume_on_retry

Conversation

@FrankYang0529

Copy link
Copy Markdown
Member

Why

TestGlueJobOperatorOpenLineageInjection.test_inject_parent_job_info_with_resume_on_retry ________________________________________________________
  providers/amazon/tests/unit/amazon/aws/operators/test_glue.py:772: in test_inject_parent_job_info_with_resume_on_retry
      glue.execute(context)
  task-sdk/src/airflow/sdk/bases/operator.py:445: in wrapper
      return func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  providers/amazon/src/airflow/providers/amazon/aws/operators/glue.py:302: in execute
      existing = self._find_job_run_id_by_task_uuid(task_uuid)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  providers/amazon/src/airflow/providers/amazon/aws/operators/glue.py:258: in _find_job_run_id_by_task_uuid
      response = self.hook.conn.get_job_runs(**request)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/client.py:602: in _api_call
      return self._make_api_call(operation_name, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/context.py:123: in wrapper
      return func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/client.py:1060: in _make_api_call
      http, parsed_response = self._make_request(
  /usr/python/lib/python3.13/site-packages/botocore/client.py:1084: in _make_request
      return self._endpoint.make_request(operation_model, request_dict)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/endpoint.py:119: in make_request
      return self._send_request(request_dict, operation_model)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/endpoint.py:197: in _send_request
      success_response, exception = self._get_response(
  /usr/python/lib/python3.13/site-packages/botocore/endpoint.py:239: in _get_response
      success_response, exception = self._do_get_response(
  /usr/python/lib/python3.13/site-packages/botocore/endpoint.py:279: in _do_get_response
      http_response = self._send(request)
                      ^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/endpoint.py:383: in _send
      return self.http_session.send(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/botocore/httpsession.py:477: in send
      urllib_response = conn.urlopen(
  /usr/python/lib/python3.13/site-packages/urllib3/connectionpool.py:789: in urlopen
      response = self._make_request(
  /usr/python/lib/python3.13/site-packages/urllib3/connectionpool.py:466: in _make_request
      self._validate_conn(conn)
  /usr/python/lib/python3.13/site-packages/urllib3/connectionpool.py:1095: in _validate_conn
      conn.connect()
  /usr/python/lib/python3.13/site-packages/urllib3/connection.py:615: in connect
      self.sock = sock = self._new_conn()
                         ^^^^^^^^^^^^^^^^
  /usr/python/lib/python3.13/site-packages/urllib3/connection.py:196: in _new_conn
      sock = connection.create_connection(
  /usr/python/lib/python3.13/site-packages/urllib3/util/connection.py:73: in create_connection
      sock.connect(sa)
  E   Failed: Timeout >60.0s

How

  • Assign glue.hook.conn before execute() and stub get_job_runs to return an empty JobRuns list.

Verification

  • uv run --project providers/amazon pytest providers/amazon/tests/unit/amazon/aws/operators/test_glue.py -q

Was generative AI tooling used to co-author this PR?
  • Yes - Claude Code

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Jul 27, 2026
@FrankYang0529
FrankYang0529 marked this pull request as ready for review July 27, 2026 08:11
@FrankYang0529
FrankYang0529 requested a review from o-nikolas as a code owner July 27, 2026 08:11
@FrankYang0529
FrankYang0529 force-pushed the airflow-flaky-test_inject_parent_job_info_with_resume_on_retry branch from 943ef86 to 4eaa594 Compare July 27, 2026 12:27
Signed-off-by: PoAn Yang <payang@apache.org>
@FrankYang0529
FrankYang0529 force-pushed the airflow-flaky-test_inject_parent_job_info_with_resume_on_retry branch from 4eaa594 to c806a5d Compare July 27, 2026 13:32
@vincbeck
vincbeck merged commit eefce6d into apache:main Jul 27, 2026
82 checks passed
@FrankYang0529
FrankYang0529 deleted the airflow-flaky-test_inject_parent_job_info_with_resume_on_retry branch July 28, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants