Update to SQLExecutor and Visualization Code Exporting #387
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* changes from perf branch to config * added flag for turning on/off lazy maintain optimization * merged in approx early pruning code * increase overall sampling start and cap * Adjust width and length criteria for early pruning vislist based on experiment results; Add warning message and test for early pruning * black version update * version lock on black * * fixed sql tests (added approx to execute constructor) * fixed sampling config test * improved Executor documentation
…ales (lux-org#262) * Add support to improve temporal action to display different timescales * Resolve PR comments * Add support to improve temporal action to display different timescales * Resolve PR comments * Reformat files using black * "All-column" vis when only few columns in dataframe lux-org#199 (lux-org#336) Co-authored-by: Caitlyn Chen <caitlynachen@berkeley.edu> Co-authored-by: Doris Lee <dorisjunglinlee@gmail.com> * documentation and cleaning * added notebook gallery * update README * removed scatterplot message in SQLExecutor * fixed typo in SQL documentation * update README and bump version * bump version * clear propagated vis data intent after PandasExecutor completes execute (lux-org#297) * fix black to stable version * Scalability: incorporate early pruning optimizations (lux-org#368) * changes from perf branch to config * added flag for turning on/off lazy maintain optimization * merged in approx early pruning code * increase overall sampling start and cap * Adjust width and length criteria for early pruning vislist based on experiment results; Add warning message and test for early pruning * black version update * version lock on black * * fixed sql tests (added approx to execute constructor) * fixed sampling config test * improved Executor documentation * timescale feature * adding weekday * adding docs * bugfix for y axis line chart export * fixing temporal axis by adding timescale variable in Clause Co-authored-by: Doris Lee <dorisjunglinlee@gmail.com> Co-authored-by: Caitlyn Chen <caitlynachen@gmail.com> Co-authored-by: Caitlyn Chen <caitlynachen@berkeley.edu>
* update binder links to jupyter lab * changed default tutorial from employee to covid dataset * added ga-beacon tracking
* don't show all column vis for series * line chart with single data point plotted as bar
* Use altair/Choropleth.py in MatpltlibRenderer.py to render choropleths; include warning * Add ._chart_scale to lux.config, accessible for AltairChart * Sync with previous upstream * Update fcode in frame * Support positive ints or floats * Change from chart_scale to plotting_scale * Apply plotting_scale to matplotlib * Black reformat * Revert matplotlib change => yields clearer output * Clean up diff * Config traitlet * black Co-authored-by: dorisjlee <dorisjunglinlee@gmail.com>
* added return HTML option to save_as_html; new docs for export * black * update github action * update python workflow * update workflow
* Use altair/Choropleth.py in MatpltlibRenderer.py to render choropleths; include warning * Docs for scale factor * Clean diff * Resolve comments
Updated code tracing to use autopep8 to properly format exported code. Updated Generalized Database Executor to handle histogram binning for relational databases that do not have width_bucket functionality.
Added proper heatmap aggregation when working with relational databases that do not have a width_bucket function. Updated the MySQL database template to include a heatmap template query.
Replaced the previous postgres only SQL executor with the GeneralDatabaseExecutor which currently has templates for both postgres and mysql
Updated SQLExecutor code to not wrap lines, this helps with the format of exported python code. Issue when exporting Pandas colored barchart aggregation code. Updated the selection of columns to be included in the final barchart to be more explicit.
Updated formatting options to clean up exported visualization generating code
Added pytest cases to test if exported python code is runnable
Fixed issue in tracing_utils where some indents were being unnecessarily removed. Updated code export tests to properly execute code
* added panel description * added semicolon to not display in notebook * refactored the Panel App title * refactor the Panel code to make it more flexible * refactor panel code to make it more flexible * remove app * replaced image with lux-resources image * save_as_html open lux link in new tab Co-authored-by: Marc Skov Madsen <masma@orsted.dk> Co-authored-by: dorisjlee <dorisjunglinlee@gmail.com>
* remove python-us dependency * black
Updated black formatting rules to exclude the Executor files. This ensures that the code lines are unwrapped, which cleans up the formatting of Vis objects' exported python code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Changed the SQLExecutor to use the more generalized template system. Instead of only working with Postgresql connections, the new SQLExecutor can be made compatible with a wider set of relational database systems as long as the appropriate query template file is provided. Currently we have also included templates for Postgres and MySQL databases.
Python code exporting has also been updated. The formatting has now been fixed using autopep8, and exported Lux Vis code can now be run without having to manually fix the code. In addition, a test suite has been added to ensure that the exported Vis code is executable.
Changes
Update to the SQLExecutor, now takes on the structure of the GeneralizedDBExecutor that we used to have. Also updated the code formatting for the Pandas and SQL Executors. This formatting removed line wrapping which helps clean up Lux code when exported.
Added test suite for both executors under the
test_export.pyfile.