Enhance DataFusion Integration with Asynchronous and Deferrable Support

[jjavieralonso]

Description

This feature enhances the DataFusion integration in Apache Airflow by adding asynchronous and deferrable support to DataFusion operators and hooks. Additionally, it introduces new DataFusion links and improves error handling.

Use case/motivation

The addition of asynchronous and deferrable support allows users to run DataFusion pipelines more efficiently by leveraging non-blocking I/O operations. This is particularly useful for long-running pipelines, reducing the need for active waiting and improving resource utilization. The new DataFusion links provide easier navigation to specific DataFusion resources within the Google Cloud Console.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

This looks like AI-generated. useless issue report that brings no value and makes no sense. We are generally blocking users that sends a lot of spam AI reports generated by bots.. as of yesterday so we will report your account and block it unless:

a) you explain how you generated reports
b) prove you are human
c) explain why you created the issue