Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: allow None as a column type #116

Merged
merged 1 commit into from
Oct 13, 2022
Merged

Conversation

betodealmeida
Copy link
Member

Tentative fix for #113.

When creating a virtual dataset from a dbt model in Snowflake we're getting the following error:

[[09:32:48]] ERROR: preset_cli.cli.superset.sync.dbt.datasets: Unable to create dataset                                                                                                           datasets.py:99
                    Traceback (most recent call last):                                                                                                                                                                              
                      File "/usr/local/lib/python3.9/site-packages/preset_cli/cli/superset/sync/dbt/datasets.py", line 97, in sync_datasets                                                                                         
                        dataset = create_dataset(client, database, model)                                                                                                                                                           
                      File "/usr/local/lib/python3.9/site-packages/preset_cli/cli/superset/sync/dbt/datasets.py", line 63, in create_dataset                                                                                        
                        return client.create_dataset(**kwargs)                                                                                                                                                                      
                      File "/usr/local/lib/python3.9/site-packages/preset_cli/api/clients/superset.py", line 534, in create_dataset                                                                                                 
                        elif column["type"].lower() == "string":                                                                                                                                                                    
                    AttributeError: 'NoneType' object has no attribute 'lower'                                  

Which suggests that when running SELECT * FROM {source} to infer the column names and types in the model at least one of the columns has the type set to None (coming from inspect.get_columns, so it's being set by the Snowflake driver).

I fixed it by checking for this case, and setting the column type to the most generic when this happens.

@betodealmeida betodealmeida changed the title fix: allow None as a column type fix: allow None as a column type Oct 13, 2022
@betodealmeida betodealmeida linked an issue Oct 13, 2022 that may be closed by this pull request
@betodealmeida betodealmeida merged commit 6b70f9c into main Oct 13, 2022
betodealmeida added a commit that referenced this pull request Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Syncing to different Snowflake databases produces an error
1 participant