Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table unable to have Airflow templated names #413

Closed
tatiana opened this issue May 26, 2022 · 0 comments · Fixed by #417
Closed

Table unable to have Airflow templated names #413

tatiana opened this issue May 26, 2022 · 0 comments · Fixed by #417
Assignees
Labels
bug Something isn't working
Milestone

Comments

@tatiana
Copy link
Collaborator

tatiana commented May 26, 2022

Describe the bug

@guohui-gao reported this issue.

Given a DAG which

  • uses one of the SQL decorators (transform, run_raw_sql or dataframe)
  • one of the arguments of this SQL decorator is a Table that has a name using an Airflow template (e.g. {{ ds }})

The user will get errors related to that table having an invalid identifier.

Example:

    tickets_daily_table = Table(name="TICKETS_{{ts_nodash}}", metadata=transform_meta)
    create_daily_table = create_daily_table(tickets_daily_table)

Version

  • Astro: 0.9.0
  • Databases: all

To Reproduce
Steps to reproduce the behavior:

  1. Checkout the branch fix-templated-table-name (f02a069) with a test that shows this issue
git checkout fix-templated-table-name
  1. Run the test
$ PYTHONPATH=`pwd`/src pytest -k test_transform_with_templated_table_name
  1. See the error
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "-": syntax error
[SQL: DROP TABLE IF EXISTS test-is-{{ test_mode }}]

Expected behavior

Users should be able to use Airflow templates in table names in astro-sdk-python SQL decorators.

@tatiana tatiana added the bug Something isn't working label May 26, 2022
@sunank200 sunank200 self-assigned this May 26, 2022
@kaxil kaxil added this to the 0.9.1 milestone May 26, 2022
@tatiana tatiana mentioned this issue May 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants