Skip to content

feat: improve lineage perf#2316

Merged
tobymao merged 1 commit intomainfrom
toby/lineage_perf
Apr 9, 2024
Merged

feat: improve lineage perf#2316
tobymao merged 1 commit intomainfrom
toby/lineage_perf

Conversation

@tobymao
Copy link
Copy Markdown
Contributor

@tobymao tobymao commented Mar 21, 2024

this should significantly increase the perf of lineage, will require sqlglot 23.1

Copy link
Copy Markdown
Contributor

@vchan vchan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tobymao
Copy link
Copy Markdown
Contributor Author

tobymao commented Mar 21, 2024

Should we trim selects here as well? https://github.com/TobikoData/sqlmesh/blob/main/web/server/api/endpoints/lineage.py#L56

no, we shouldn't because you use the output to generate a UI and we want those to look clean. but this breaks some tests so i'll need some time to fix it and test

@tobymao tobymao force-pushed the toby/lineage_perf branch 5 times, most recently from d94a220 to abeda2a Compare April 9, 2024 19:07
Copy link
Copy Markdown
Collaborator

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, only some questions FMI.

@tobymao tobymao force-pushed the toby/lineage_perf branch from abeda2a to 37907d8 Compare April 9, 2024 20:43
@tobymao tobymao enabled auto-merge (squash) April 9, 2024 20:43
@tobymao tobymao merged commit 59c585f into main Apr 9, 2024
@tobymao tobymao deleted the toby/lineage_perf branch April 9, 2024 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants