Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

let user choose whether to filter temp table or not #23

Closed
reata opened this issue Aug 11, 2019 · 3 comments · Fixed by #56
Closed

let user choose whether to filter temp table or not #23

reata opened this issue Aug 11, 2019 · 3 comments · Fixed by #56
Assignees
Labels
bug Something isn't working enhancement New feature or request
Milestone

Comments

@reata
Copy link
Owner

reata commented Aug 11, 2019

Currently, If a table is both source table and target table. sqllineage will identify this table as temp table and hide it from user.

We should give use control over the display of temp table

@reata reata added the enhancement New feature or request label Aug 11, 2019
@reata reata added this to the v0.2.0 milestone Aug 11, 2019
@reata reata self-assigned this Aug 11, 2019
@reata reata added this to To do in SQLLineage v0.2.0 via automation Aug 11, 2019
@reata
Copy link
Owner Author

reata commented Nov 17, 2019

insert overwrite table tab_a
select col_a from tab_b
inner join tab_a
on tab_b.x = tab_a.x

here tab_a is not temp table at all, and yet it's filtered.
This is also used to get data from last partition of tab_a, adding some filter logic, and then insert into the new partition.

@reata reata added the bug Something isn't working label Nov 17, 2019
@reata
Copy link
Owner Author

reata commented Nov 17, 2019

create table if not exists tab_a as
select * from tab_b;
create table if not exists tab_c as 
select * from tab_a;

Here tab_a may not be temp table also.
This temp table checking method is deeply flawed. We should discuss whether to keep it. Even if we decide to keep, this should default be turned off.

This was referenced Nov 18, 2019
@reata
Copy link
Owner Author

reata commented Apr 11, 2020

We should add a combiner abstract layer. The parsing result remain at Statement level. And temp table logic is left for combiner to deal with.

@reata reata modified the milestones: v0.2.0, v0.3.0 Apr 11, 2020
@reata reata added this to To do in SQLLineage v0.3.0 via automation Apr 11, 2020
@reata reata removed this from To do in SQLLineage v0.2.0 Apr 11, 2020
@reata reata closed this as completed in #56 Jul 18, 2020
SQLLineage v0.3.0 automation moved this from To do to Done Jul 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

1 participant