Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make database & schema optional in SQL files when using aql.render() #124

Closed
tatiana opened this issue Feb 21, 2022 · 0 comments · Fixed by #144
Closed

Make database & schema optional in SQL files when using aql.render() #124

tatiana opened this issue Feb 21, 2022 · 0 comments · Fixed by #144
Assignees
@dimberman
Labels
bug Something isn't working priority/high High priority
Milestone
0.7.x

Comments

@tatiana
Copy link
Collaborator

tatiana commented Feb 21, 2022 •
edited

Context

At the moment, database and schema need to be specified in all SQL files used by aql.render(); otherwise, Astro (0.5.1) raises the following exception:

  File "/usr/local/lib/python3.9/site-packages/astro/utils/snowflake_transform.py", line 30, in _handle_table
        return t.database + "." + t.schema + "." + t.table_name
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

This means that the user has to specify the database and schema in all upstream and downstream tasks.

Acceptance criteria

  • Evaluate the possible ways of solving this problem, and chose an approach. Some possible solutions could involve using a configuration file, an environment variable, or inheriting from the upstream task.
  • Astro users are able to use aql.render without specifying database and schema headers in all SQL files.
  • Ask @mag3141592 (bug reporter) to review the PR
@tatiana tatiana added the bug Something isn't working label Feb 21, 2022
@tatiana tatiana added this to the 0.6.1 milestone Feb 21, 2022
@tatiana tatiana added the priority/high High priority label Feb 21, 2022
dimberman added a commit that referenced this issue Feb 25, 2022
@dimberman
Fix inheritance from partially defined output tables #124
50b675c 
This fixes an issue where the sql decorator doesn't pass on credentials
for partially defined or undefined output tables.
@dimberman dimberman self-assigned this Feb 25, 2022
@dimberman dimberman linked a pull request Feb 25, 2022 that will close this issue
#124 Fix inheritance from partially defined output tables #144
Merged
dimberman added a commit that referenced this issue Feb 28, 2022
@dimberman
#124 Fix inheritance from partially defined output tables (#144)
e7b4285 
* Fix inheritance from partially defined output tables #124

This fixes an issue where the sql decorator doesn't pass on credentials
for partially defined or undefined output tables.

* also ensures that partial tables are handled for dataframe functions

* cleanup

* cleanup
@dimberman dimberman mentioned this issue Feb 28, 2022
Closed
utkarsharma2 pushed a commit that referenced this issue Mar 30, 2022
@dimberman @utkarsharma2
#124 Fix inheritance from partially defined output tables (#144)
85909c4 
* Fix inheritance from partially defined output tables #124

This fixes an issue where the sql decorator doesn't pass on credentials
for partially defined or undefined output tables.

* also ensures that partial tables are handled for dataframe functions

* cleanup

* cleanup
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dimberman dimberman

Labels
bug Something isn't working priority/high High priority
Projects
None yet
Milestone
0.7.x
Development

Successfully merging a pull request may close this issue.

#124 Fix inheritance from partially defined output tables astronomer/astro-sdk
2 participants
@tatiana @dimberman