Python: extract statement granularity info

[user5651]

I've run:

from sqllineage.runner import LineageRunner

result=LineageRunner(sql)

sql is a string containing multiple SQL statements.

I want to extract the source and target table for each statement. Using the command line this is printed nicely, for example:

...
Statement #97: CREATE OR REPLACE TABLE <DATASET>.<SOURCE_TABLE>...
    table read: [Table: <DATASET>.<SOURCE_TABLE>]
    table write: [Table: <DATASET>.<TARGET_TABLE>]
    table rename: []
    table drop: []
    table intermediate: []
...

How can I extract this info cleanly from the result variable in python? Right now I can only extract the source (result.source_tables) and target tables (result.target_tables) for all statements combined rather than individually.

Info: python3.7, sqllineage==1.1.4.

[reata]

Since this is more of a "how-to" question, I converted this from issue to discussion.

For your question, two options here:

1. To split the sql into multiple string-represented statements first and then feed each statement string to LineageRunner one by one. This is the preferred way to do it.

import sqlparse
from sqllineage.runner import LineageRunner
for statement in sqlparse.split(sql):
    result = LineageRunner(statement)
    print(result.source_tables, result.target_tables)
    # or if you want the text result, just print(result)

2. To use sqllineage's internal API to read the same result as you have with verbose option passed in LineageRunner. I'd advice against it since this internal API is subject to change in future release. But if you insist, here's the way:

from sqllineage.runner import LineageRunner
result = LineageRunner(sql)
result._eval()
for lineage_result in result._lineage_results:
    print(lineage_result.read, lineage_result.write, lineage_result.rename, lineage_result.drop, lineage_result.intermediate)
    # or if you want the text result, just print(lineage_result)

Let me know if you have further questions.

[user5651]

Amazing, thanks @reata!