Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAP-1012] Support for new agate data type in Spark #935

Closed
gshank opened this issue Nov 7, 2023 · 0 comments · Fixed by #936
Closed

[ADAP-1012] Support for new agate data type in Spark #935

gshank opened this issue Nov 7, 2023 · 0 comments · Fixed by #936

Comments

@gshank
Copy link
Contributor

gshank commented Nov 7, 2023

dbt Core changes

To support integers in the "show" command, a new agate data_type, Integer, was added in dbt Core in core/dbt/clients/agate_helper.py. The agate table creation code is also used when creating empty tables for seeds, such as when a seed file has headers but doesn't have any rows. Because the "convert_agate_type" method did not include the new Integer type, "None" was returned and the sql generated by the macro "create_csv_table" was incorrect.

dbt Core pull request dbt-labs/dbt-core#9004 adds a new "convert_integer_type" method and adds Integer to the list of agate types in "convert_agate_type".

Adapter changes

This pull request will pull in the new empty seed test and provide a "convert_integer_type" method.

@github-actions github-actions bot changed the title Support for new agate data type in Spark [ADAP-1012] Support for new agate data type in Spark Nov 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant