-
Notifications
You must be signed in to change notification settings - Fork 13.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(dashboard): Let users download full CSV of a table #15046
feat(dashboard): Let users download full CSV of a table #15046
Conversation
- Add menu option to download full CSV. Probably will change it
Exporting a full CSV could cause the server to run out of memory and/or compute, how should this be handled? I think and ideal solution would be to stream the results down to the client, but that would be a significantly larger effort. Perhaps adding a config key to optionally enable this feature might be enough to mitigate this concern. |
thanks for the contribution! @rusackas this dropdown list is getting a bit long, we may need to restructure it a bit. |
100% agree with the perf question, in this case i think a feature flag/config option would resolve this. Streaming results seems especially difficult to do with our current infra (maybe easier once async everywhere is ready for primetime). For context for our use case, we're ok with the performance issues and potential OOMs that this might introduce since everything is in k8s and autoscalable anyway. |
Codecov Report
@@ Coverage Diff @@
## master #15046 +/- ##
==========================================
- Coverage 77.54% 77.26% -0.28%
==========================================
Files 967 969 +2
Lines 49752 49918 +166
Branches 6351 6393 +42
==========================================
- Hits 38578 38569 -9
- Misses 10972 11146 +174
- Partials 202 203 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
@@ -87,7 +87,8 @@ const SHOULD_UPDATE_ON_PROP_CHANGES = Object.keys(propTypes).filter( | |||
); | |||
const OVERFLOWABLE_VIZ_TYPES = new Set(['filter_box']); | |||
const DEFAULT_HEADER_HEIGHT = 22; | |||
|
|||
// superset backend supports upto a million rows | |||
const MAXIMUM_NUMBER_OF_ROWS_IN_CSV = 1000000; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a little concern about too many configs in different places. Is possible to read this parameter from backend and carry it to frontend as part of bootstrap data. check this place:
superset/superset/views/base.py
Line 84 in cc2b4fe
FRONTEND_CONF_KEYS = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
* - Convert SliceHeader to TSX in progress - Add menu option to download full CSV. Probably will change it * Add Download Full CSV feature, and tests * Added more tests, more TS fixes * Added feature flag * Update @superset-ui package versions * Update @superset-ui packages versions * use backend config instead of hardcoding number of rows * Update tests * front end test fix * Lint fixes and test fixes
* - Convert SliceHeader to TSX in progress - Add menu option to download full CSV. Probably will change it * Add Download Full CSV feature, and tests * Added more tests, more TS fixes * Added feature flag * Update @superset-ui package versions * Update @superset-ui packages versions * use backend config instead of hardcoding number of rows * Update tests * front end test fix * Lint fixes and test fixes
* - Convert SliceHeader to TSX in progress - Add menu option to download full CSV. Probably will change it * Add Download Full CSV feature, and tests * Added more tests, more TS fixes * Added feature flag * Update @superset-ui package versions * Update @superset-ui packages versions * use backend config instead of hardcoding number of rows * Update tests * front end test fix * Lint fixes and test fixes
SUMMARY
Lets user download the full CSV of a table.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
Manual and CI
ADDITIONAL INFORMATION