Convert df to pyspark DataFrame if it is koalas before writing

[dbeatty10]

resolves #320

Description

If the DataFrame to materialize as a table has type databricks.koalas.frame.DataFrame, then convert it to a pyspark.sql.dataframe.DataFrame.

Group of related pull requests

• Test for Koalas DataFrames dbt-core#5928
• Convert df to pyspark DataFrame if it is koalas before writing #321 (this one)
• Convert df to pyspark DataFrame if it is koalas before writing dbt-spark#474

Checklist

• I have read the contributing guide and understand what's expected of me
• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
• I have run changie new to create a changelog entry

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the dbt-bigquery contributing guide.

[dbeatty10]

Did only manual testing for this. It worked.

Only tested on Spark 3.2.2. Ideally, this would be tested on Spark 3.1 also.

Here was the dbt Python model used for testing:

import databricks.koalas as ks


def model(dbt, session):
    dbt.config(
        materialized="table",
    )

    df = ks.DataFrame(
        {'City': ['Buenos Aires', 'Brasilia', 'Santiago', 'Bogota', 'Caracas'],
        'Country': ['Argentina', 'Brazil', 'Chile', 'Colombia', 'Venezuela'],
        'Latitude': [-34.58, -15.78, -33.45, 4.60, 10.48],
        'Longitude': [-58.66, -47.91, -70.66, -74.08, -66.86]}
        )

    return df

Used the following code to report the version of Spark (which came from here):

import pyspark
from pyspark.sql import SparkSession

# Create SparkSession
spark = SparkSession.builder.master("local[1]") \
                    .appName('SparkByExamples.com') \
                    .getOrCreate()

# PySpark Version 3.2.2 and 3.2.2
msg = f"PySpark Version {spark.version} and {spark.sparkContext.version}"
raise Exception(msg)