New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add quoting support #65
Conversation
@cstruct could you fix the linting issues? |
Yes, of course! I'll fix it tomorrow when I'm home again. |
Add quoting support both where Presto and Hive style is used.
Sorry for the wait! Forgot to do it during the weekend :) |
@cstruct I tested this PR, and somehow this changes breaks our iceberg implementation. Here the error log that I'm getting:
For what I can see the first part where it breaks is the drop staement.
that should be rendered as |
@nicor88 I'm not familiar with the iceberg implementation but I pushed a commit with a guess for a fix. Could you try again? If this doesn't work I'll have to create a test setup and do some proper debugging. |
@cstruct we can help you out, most of the code is in helpers_iceberg and in incremental (not sure if it's needed here, it might be needed too). As you can see we have a custom drop for iceberg, as we don't use the general drop overwrite, the reason behind is that a drop table in iceberg lead to actual data deletion in S3, but this is not valid in the other cases. |
@nicor88 alright it doesn't seem to involved. Unfortunately I've managed to bork my local python env, might have time to fix and test later today otherwise later this week. For the time being I pushed a commit that changes to Hive style where dropping and creating external tables in helpers_iceberg which I know are required. If you have the time and feel for it you can give it a try again otherwise I'll look into it later. |
For those that are interested the reason this broke support for iceberg tables is that previously |
@cstruct any plan to move this forward? |
Please get the latest changes from |
Yes, I do plan to revisit this PR when I have the time and energy to do so, I can't give any guarantees as to when that will happen though. If you feel that you do not want to wait for that, the |
What is still to be done for this PR to be merged? Are the latest changes to helpers_iceberg tested @nicor88? I'm willing to continue the work on this if need be. Just not sure if the iceberg tests are still failing or not.. |
@BrechtDeVlieger |
Add quoting support both where Presto and Hive style is used.
From request by @nicor88