Add local dataframe to delta support

[dimberman]

Please describe the feature you'd like to see

I'd like to see a feature that allows users to take a local dataframe and send it to a Databricks Delta table. This would enable us to more easily and efficiently load data into Databricks for processing and analysis.

Describe the solution you'd like

A function or method that takes a local dataframe and a connection string for a Databricks Delta table as input, and loads the data from the dataframe into the table. It would be helpful if the function also had options for specifying the load behavior (e.g. append vs. overwrite).

Here's an example of what the function signature might look like:

@aql.dataframe()
def df_func() -> pandas.Dataframe:
    return df
    
with dag:
    df_func(output_table=Table(conn_id="my_delta_conn")

Are there any alternatives to this feature?
One alternative would be to use the Databricks API to load data into a Delta table. This would require users to manually construct the API request and handle any errors that might occur, whereas the proposed function would handle these details internally.

Additional context
This feature will be released as part of the 0.1 release so users can start testing basic functionality.

Acceptance Criteria

• All checks and tests in the CI should pass
• Unit tests (90% code coverage or more, once available)
• Integration tests (if the feature relates to a new database or external service)
• Example DAG
• Docstrings in reStructuredText for each of methods, classes, functions and module-level attributes (including Example DAG on how it should be used)
• Exception handling in case of errors
• Logging (are we exposing useful information to the user? e.g. source and destination)
• Improve the documentation (README, Sphinx, and any other relevant)
• How to use Guide for the feature (example)