chore: update fixtures scope that are loading data into the analytical db #17780
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bkyryliuk
force-pushed
the
bogdan/change_fixture_scope
branch
4 times, most recently
from
179755c
to
07a5ed7
Compare
bkyryliuk
force-pushed
the
bogdan/change_fixture_scope
branch
from
07a5ed7
to
6854d0d
Compare
bkyryliuk
force-pushed
the
bogdan/change_fixture_scope
branch
5 times, most recently
from
f341163
to
1fde0ce
Compare
bkyryliuk
force-pushed
the
bogdan/change_fixture_scope
branch
from
1fde0ce
to
e5b521e
Compare
Codecov Report
@@ Coverage Diff @@
## master #17780 +/- ##
==========================================
+ Coverage 67.72% 67.79% +0.07%
==========================================
Files 1605 1605
Lines 64205 64207 +2
Branches 6790 6790
==========================================
+ Hits 43481 43529 +48
+ Misses 18868 18822 -46
Partials 1856 1856
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
bkyryliuk
changed the title
bkyryliuk
requested review from
betodealmeida,
villebro,
amitmiran137 and
suddjian
and removed request for
villebro
betodealmeida
approved these changes
Dec 16, 2021
shcoderAlex
pushed a commit
to casual-precision/superset
that referenced
this pull request
Feb 7, 2022
…l db (apache#17780) * Update db setup fixture scope * Load data into db only once * Update fixture scopes for data loading * cleanup imports * try import * Update scope * fix cleanup * Import all fixtures * Separate data log and metadata creation * Retain table description * fix dtype Co-authored-by: Bogdan Kyryliuk <bogdankyryliuk@dropbox.com>
bwang221
pushed a commit
to casual-precision/superset
that referenced
this pull request
Feb 10, 2022
…l db (apache#17780) * Update db setup fixture scope * Load data into db only once * Update fixture scopes for data loading * cleanup imports * try import * Update scope * fix cleanup * Import all fixtures * Separate data log and metadata creation * Retain table description * fix dtype Co-authored-by: Bogdan Kyryliuk <bogdankyryliuk@dropbox.com>
mistercrunch
added
🏷️ bot
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
Play with fixture scopes to see if tests will be faster
Hive and presto DBs are slow, we should load test data only once rather than do it for every module / test function
Speed improvements:
Python Presto/Hive / test-postgres-presto (3.8) 38m vs 92 m
Python Presto/Hive / test-postgres-hive 39m vs 53 m
There are probably other tests as well that can benefit from a similar improvement