Skip to content

Conversation

@amoghrajesh
Copy link
Contributor

Fixing static check failure in main:

Validate provider.yaml files.....................................................Failed
- hook id: check-provider-yaml-valid
- exit code: 1
  Using 'uv' to install Airflow


  Using airflow version from current sources

  /opt/airflow/airflow-core/src/airflow/dag_processing/dagbag.py:40 DeprecationWarning: airflow.exceptions.AirflowDagCycleException is deprecated. Use airflow.sdk.exceptions.AirflowDagCycleException instead.
  Verifying packages on Architecture.ARM architecture. Platform: aarch64.
  Traceback (most recent call last):
    File "/opt/airflow/scripts/in_container/run_provider_yaml_files_check.py", line 733, in <module>
      all_parsed_yaml_files: dict[str, dict] = _load_package_data(paths)
    File "/opt/airflow/scripts/in_container/run_provider_yaml_files_check.py", line 134, in _load_package_data
      with open(provider_yaml_path) as yaml_file:
  FileNotFoundError: [Errno 2] No such file or directory: '/opt/airflow/airbyte/provider.yaml'

  Error 1 returned
  /opt/hostedtoolcache/Python/3.10.19/arm64/lib/python3.10/tempfile.py:869: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp2a48q81w'>
    _warnings.warn(warn_message, ResourceWarning)

When the pre-commit hook runs from the providers/.pre-commit-config.yaml file, it passes file paths relative to the providers/ directory (e.g., airbyte/provider.yaml). However, the script run_provider_yaml_files_check.py inside breeze expects paths relative to root directory. This caused it to look for /opt/airflow/airbyte/provider.yaml instead of /opt/airflow/providers/airbyte/provider.yaml

In breeze shell:

root@4548c6611747:/opt/airflow# ls /opt/airflow/airbyte/provider.yaml
ls: cannot access '/opt/airflow/airbyte/provider.yaml': No such file or directory
root@4548c6611747:/opt/airflow# ls /opt/airflow/providers/airbyte/provider.yaml
/opt/airflow/providers/airbyte/provider.yaml

As a solution for the time being, modified script to prepend providers/ to all file paths before passing them to the breeze container.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@boring-cyborg boring-cyborg bot added area:dev-tools backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch labels Oct 26, 2025
@potiuk
Copy link
Member

potiuk commented Oct 26, 2025

I don't think it's a complete solution - I found that this provider.yaml was broken for a short while and it needs more fixes.

@potiuk
Copy link
Member

potiuk commented Oct 26, 2025

I am working on a better fix.

@jscheffl
Copy link
Contributor

Uh, oh, sorry, my fault due to re-structuring :-(
Interesting, I thought this would have been tested with the PR making the change? Are not all static checks being executed?

@potiuk
Copy link
Member

potiuk commented Oct 26, 2025

No No.. It's good -> the problem was that it was broken since moving providers to "providers" folder.

@potiuk
Copy link
Member

potiuk commented Oct 26, 2025

Complete fix in #57283

@potiuk potiuk closed this Oct 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants