Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different function and terminal outputs on "with as" #452

Closed
ArtemSamoilov opened this issue Sep 13, 2023 · 2 comments
Closed

Different function and terminal outputs on "with as" #452

ArtemSamoilov opened this issue Sep 13, 2023 · 2 comments
Labels
question Further information is requested

Comments

@ArtemSamoilov
Copy link

Hello reata!

When I use "WITH AS" in my query the output from using a terminal command "sqllineage -f file.sql -l column" is different from the output of get_sql_lineage() or print_column_lineage().

Here is the query:
INSERT INTO target_db.target_table (
id
, name
, age
, email
)
WITH info AS {
SELECT id
, name
, age
, email
FROM source_db.source_table
WHERE age >= 25
)
SELECT id
, name
, age
, email
FROM info
GROUP BY 1, 2, 3, 4;

Terminal output (correct):
target_db.target_table.age <- info.age <- source_db.source_table.age
target_db.target_table.email <- info.email <- source_db.source_table.email
target_db.target_table.id <- info.id <- source_db.source_table.id
target_db.target_table.name <- info.name <- source_db.source_table.name

get_column_lineage() returns empty brackets.

I would appreciate a lot if you look into this issue.

@reata
Copy link
Owner

reata commented Sep 20, 2023

from sqllineage.runner import LineageRunner

sql = """INSERT INTO target_db.target_table (
id
, name
, age
, email
)
WITH info AS (
SELECT id
, name
, age
, email
FROM source_db.source_table
WHERE age >= 25
)
SELECT id
, name
, age
, email
FROM info
GROUP BY 1, 2, 3, 4;"""

LineageRunner(sql).get_column_lineage()

output:

[(Column: source_db.source_table.age,
  Column: info.age,
  Column: target_db.target_table.age),
 (Column: source_db.source_table.email,
  Column: info.email,
  Column: target_db.target_table.email),
 (Column: source_db.source_table.id,
  Column: info.id,
  Column: target_db.target_table.id),
 (Column: source_db.source_table.name,
  Column: info.name,
  Column: target_db.target_table.name)]

Can you share more details, like python version/sqllineage version/exact code you're using?

Based on the code I used above, together with Py38 and sqllineage v1.4.7, the python API output is the same as terminal.

@reata reata added the question Further information is requested label Sep 20, 2023
@ArtemSamoilov
Copy link
Author

Thanks a lot! Updating the version did the trick.

@reata reata closed this as completed Sep 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants