Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(sqllab): reduce bootstrap data delay by queries #27488

Merged

Conversation

justinpark
Copy link
Member

SUMMARY

The SQL Lab bootstrap data, which includes all queries associated with every open editor tab, is quite large in size.
This has been causing a noticeable delay in the initial loading time for SQL Lab. It's important to note that these queries are solely used for maintaining a query history, so this update is designed to bypass these queries in the bootstrap data.
Instead, we will use the Query API to fetch the same results when the Query History component loads, making the process more efficient.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Before:
Superset-2

After:
Screenshot 2024-03-11 at 3 38 15 PM

  • Paginated /w infinite scrolling
paginated-editor-queries.mov

TESTING INSTRUCTIONS

Go to SQL Lab and run some queries in a tab and then switch the query history tab

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@github-actions github-actions bot added the api Related to the REST API label Mar 13, 2024
@justinpark justinpark force-pushed the perf--migrate-queries-from-initial-state branch from cb67955 to 17be1ac Compare March 13, 2024 03:26
Copy link

codecov bot commented Mar 13, 2024

Codecov Report

Attention: Patch coverage is 85.71429% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 69.74%. Comparing base (735b895) to head (050202e).
Report is 1 commits behind head on master.

Files Patch % Lines
...ntend/src/SqlLab/components/QueryHistory/index.tsx 73.68% 3 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #27488      +/-   ##
==========================================
+ Coverage   69.73%   69.74%   +0.01%     
==========================================
  Files        1909     1910       +1     
  Lines       74692    74719      +27     
  Branches     8325     8335      +10     
==========================================
+ Hits        52086    52115      +29     
  Misses      20556    20556              
+ Partials     2050     2048       -2     
Flag Coverage Δ
hive 53.73% <100.00%> (+0.02%) ⬆️
javascript 57.26% <84.37%> (+0.04%) ⬆️
mysql 78.01% <100.00%> (-0.01%) ⬇️
postgres 78.11% <100.00%> (-0.01%) ⬇️
presto 53.68% <100.00%> (+0.02%) ⬆️
python 83.14% <100.00%> (-0.01%) ⬇️
sqlite 77.55% <100.00%> (-0.01%) ⬇️
unit 56.64% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@justinpark
Copy link
Member Author

/testenv up FEATURE_SQLLAB_BACKEND_PERSISTENCE=true

Copy link
Contributor

@justinpark Ephemeral environment spinning up at http://35.87.7.33:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

@justinpark justinpark added the v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch label Mar 13, 2024
@michael-s-molina michael-s-molina removed the v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch label Mar 14, 2024
Copy link
Member

@michael-s-molina michael-s-molina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@justinpark justinpark merged commit f4bdcb5 into apache:master Mar 18, 2024
38 checks passed
Copy link
Contributor

Ephemeral environment shutdown and build artifacts deleted.

deepak-kumar-007 pushed a commit to deepak-kumar-007/superset-forked that referenced this pull request Mar 18, 2024
@michael-s-molina michael-s-molina added the v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch label Mar 19, 2024
michael-s-molina pushed a commit that referenced this pull request Mar 19, 2024
sfirke pushed a commit to sfirke/superset that referenced this pull request Mar 22, 2024
EandrewJones pushed a commit to UMD-ARLIS/superset that referenced this pull request Apr 5, 2024
@mistercrunch mistercrunch added 🍒 4.0.0 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels labels Apr 17, 2024
qleroy pushed a commit to qleroy/superset that referenced this pull request Apr 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Related to the REST API 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XL v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch 🍒 4.0.0 🍒 4.0.1 🍒 4.0.2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants