Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with external materialized #2

Closed
pyjads opened this issue Feb 16, 2024 · 3 comments
Closed

Issue with external materialized #2

pyjads opened this issue Feb 16, 2024 · 3 comments

Comments

@pyjads
Copy link

pyjads commented Feb 16, 2024

{{ config(
    materialized='external',
    plugin = 'delta',
    location = '/Users/pyjads/dbt_trial/store/dest',
    mode = "overwrite",
    storage_options = {
        "connection_string": "DefaultEndpointsProtocol=https;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;EndpointSuffix=core.windows.net",
    }
) }}

select * from {{ source('external-source','userdata1')}}

I get following error
IO Error: Cannot open file "/Users/pyjads/dbt_trial/store/dest": Is a directory

If I am not mistaken, the following script should create the deltalake table at the location.

@milicevica23
Copy link
Collaborator

Hi @pyjads,
You should be able to set instead "external" -> "external_table", as here in the example https://github.com/l-mds/demo-dbt-duckdb-delta-plugin/blob/main/dbt_duckdb_delta/models/write/customer_overwrite_azure.sql

more about the problem can be found here #1
or read here duckdb/dbt-duckdb#284

Ultimately this should be possible but currently, i have to refactor the external materialization first

@pyjads
Copy link
Author

pyjads commented Feb 17, 2024

The above code works now with 'external', after installing the respository from the branch milicevica23:feature/support-delta-plugin-write.

Thank you.

@milicevica23
Copy link
Collaborator

ah yes, in the current dbt-duckdb is just the read part
if you push to the azure then please use external_table materialization for now

i would close this issue, feel free to open another one if you find some problems

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants