Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ELE-127] Baseline schema changes duration is very long #541

Open
elongl opened this issue Jan 9, 2023 · 2 comments
Open

[ELE-127] Baseline schema changes duration is very long #541

elongl opened this issue Jan 9, 2023 · 2 comments
Labels
dbt package Enhancement New feature or request

Comments

@elongl
Copy link
Member

elongl commented Jan 9, 2023

Thread link

image.png

From SyncLinear.com | ELE-127

@Maayan-s Maayan-s assigned ghost and unassigned ghost Jan 9, 2023
@elongl elongl added the Enhancement New feature or request label Jan 22, 2023
@elongl elongl self-assigned this Jan 29, 2023
@FridayPush
Copy link

Ran into the same issue when implementing the schema change tests, non baseline version. Using Redshift as our DWH. A new source was added outside of our teams chage management workflows so figured the schema anomalies would be nice.

Approximately 40 tables were added with schema changes, and each table introduced 1-1.5minutes of runtime. Our total job runtime jumped by 15 minutes. I don't know what ability Elementary has to pull 'table' tests out to a larger scope but perhaps the schema change anomalies running as a posthook, where information schema only needs to be queried once might be helpful. For now we're going to remove the tests.

@Maayan-s
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbt package Enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants