Skip to content

Fix Alibaba OSS provider: configurable endpoint and task handler log …#66479

Open
hasangzc wants to merge 1 commit intoapache:mainfrom
hasangzc:fix-alibaba-oss-endpoint-and-task-handler-read
Open

Fix Alibaba OSS provider: configurable endpoint and task handler log …#66479
hasangzc wants to merge 1 commit intoapache:mainfrom
hasangzc:fix-alibaba-oss-endpoint-and-task-handler-read

Conversation

@hasangzc
Copy link
Copy Markdown

@hasangzc hasangzc commented May 6, 2026

Summary

  • OSSHook._get_client(): The endpoint was hardcoded to the public URL
    (oss-{region}.aliyuncs.com), making it impossible to use VPC/internal
    endpoints (e.g. oss-eu-central-1-internal.aliyuncs.com). Now reads from
    the connection's extra config with endpoint key, falling back to the
    public default.

  • OSSTaskHandler._read(): Called self.oss_log_exists() and self.oss_read()
    which don't exist on OSSTaskHandler. These methods live on OSSRemoteLogIO
    and must be accessed via self.io. This caused the webserver to fail silently
    when reading logs from OSS.

Test plan

  • Added test_get_client_uses_default_endpoint - verifies default endpoint behavior
  • Added test_get_client_uses_custom_endpoint_from_connection - verifies custom endpoint from extra config
  • Added test_read_calls_oss_methods_via_io - verifies _read() uses self.io
  • Added test_read_falls_back_to_local_when_no_remote_log - verifies local fallback

Was generative AI tooling used to co-author this PR?
  • Yes

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg
Copy link
Copy Markdown

boring-cyborg Bot commented May 6, 2026

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@hasangzc hasangzc force-pushed the fix-alibaba-oss-endpoint-and-task-handler-read branch from 5d88022 to 2b11c34 Compare May 6, 2026 22:15
…reading

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant