Update to executor documentation. #363
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Uses unique value metadata to verify if a value is valid
frame.py was trying to import luxWidget instead of luxwidget
… with SQL Executor Some interestingness functions required the number of observations in the data and visualization, so I added these values to the metadata to make the scoring work when using the SQL executor Added tests for SQL executor
Removed lines that changed Year column type to datetime
SQL Executor tests interfering with travis build, commenting out for now
…nto Database-Executor
Issue where validator was relying on metadata which was not yet generated, moved metadata calculation before validation step in frame.py
Renamed num_obs to length, removed ordinal variable from Executor mapping function
* Merging Recent SQL Executor changes * Fix to Validator Uses unique value metadata to verify if a value is valid * Fix Bug with Widget Rendering frame.py was trying to import luxWidget instead of luxwidget * Added Number of Observations to MetaData, Fixed Interestingness issue with SQL Executor Some interestingness functions required the number of observations in the data and visualization, so I added these values to the metadata to make the scoring work when using the SQL executor Added tests for SQL executor * Re-added Licensing Headers * Adding Recent frame.py changes * Adjusted SQL Executor Tests Removed lines that changed Year column type to datetime * Update Frame with new Action Registering * Resolving Conflicts in frame.py * Commenting out local SQL Executor tests SQL Executor tests interfering with travis build, commenting out for now * Update correlation.py * Update frame.py * Fixing Code Format * Cleaning up Pandas Executor imports * Fix Validation Bug Issue where validator was relying on metadata which was not yet generated, moved metadata calculation before validation step in frame.py * Changed metadata variable name Renamed num_obs to length, removed ordinal variable from Executor mapping function Co-authored-by: 19thyneb <thyne.boonmark@gmail.com> Co-authored-by: Doris Lee <dorisjunglinlee@gmail.com>
Updated travis.yml file to create postgresql database in test instance. Added script to populate test database with data.
Updated database credentials
* Merging Recent SQL Executor changes * Fix to Validator Uses unique value metadata to verify if a value is valid * Fix Bug with Widget Rendering frame.py was trying to import luxWidget instead of luxwidget * Added Number of Observations to MetaData, Fixed Interestingness issue with SQL Executor Some interestingness functions required the number of observations in the data and visualization, so I added these values to the metadata to make the scoring work when using the SQL executor Added tests for SQL executor * Re-added Licensing Headers * Adding Recent frame.py changes * Adjusted SQL Executor Tests Removed lines that changed Year column type to datetime * Update Frame with new Action Registering * Resolving Conflicts in frame.py * Commenting out local SQL Executor tests SQL Executor tests interfering with travis build, commenting out for now * Update correlation.py * Update frame.py * Fixing Code Format * Cleaning up Pandas Executor imports * Fix Validation Bug Issue where validator was relying on metadata which was not yet generated, moved metadata calculation before validation step in frame.py * Changed metadata variable name Renamed num_obs to length, removed ordinal variable from Executor mapping function * Added script to generate Postgresql database Updated travis.yml file to create postgresql database in test instance. Added script to populate test database with data. * Update upload_car_data.py Updated database credentials * Updated script name in travis.yml * Removed unnecessary import from travis.yml * Added psycopg2 to requirements.txt * Creating Postgres test database in travis * Fixed directory issue Co-authored-by: 19thyneb <thyne.boonmark@gmail.com> Co-authored-by: Doris Lee <dorisjunglinlee@gmail.com>
Added tests for basic SQL Executor functionality.
Added an example notebook to showcase how to use the sql-engine. Fixed variable reference in interestingness.py that was causing issues.
Aiming to simplify the initial Lux installation. Will include a notice in the SQL documentation letting users know that they will have to install the library themselves if the want to use the LuxSQLTable functionality.
Removed psycopg2 from Lux requirements
…nto Database-Executor
Merging in master branch changes
Separated and moved the SQLExecutor tests to the tests_sql folder. Updated the make file to include a command to test both the pandas and SQL functionalities. Updated the github workflow to also run the tests_sql folder
Separated and moved the SQLExecutor tests to the tests_sql folder. Updated the make file to include a command to test both the pandas and SQL functionalities. Updated the github workflow to also run the tests_sql folder
Fixing merge conflicts with Master branch
Need to set the executor type to "Pandas" at the start of the PandasExecutor test suite. Fixed column name references in the SQLExecutor tests
1. Added documentation for SQL Execution Engine. 2. Added SQL Executor gif in images folder.
Sql Executor documentation
Codecov Report
@@ Coverage Diff @@
## master #363 +/- ##
==========================================
+ Coverage 84.32% 84.39% +0.07%
==========================================
Files 51 51
Lines 3891 3891
==========================================
+ Hits 3281 3284 +3
+ Misses 610 607 -3
Continue to review full report at Codecov.
|
dorisjlee
requested changes
Apr 16, 2021
doc/source/advanced/executor.rst
Outdated
| Lux extends its visualization exploration operations to data within SQL databases. By using the SQL Executor, users can specify a SQL database to connect a LuxSQLTable for generating all the visualizations recommended in Lux. | ||
| Lux extends its visualization capabilities to SQL within Postgres databases. By using the SQLExecutor, users can create a :code:`LuxSQLTable` that connects to a Postgres database. When the :code:`LuxSQLTable` object is printed out, Lux displays a subset of the data and recommends a default set of visualizations to display. | ||
|
|
||
| .. image:: https://github.com/lux-org/lux/blob/sql-engine/doc/source/img/SQLexecutor.gif?raw=true |
There was a problem hiding this comment.
@thyneb19: Can we remove SQLexecutor.gif from this PR and add it into lux-resources and link it here? That prevents us from adding large files into the main repo.
There was a problem hiding this comment.
@NiStannum: Could we do a new recording of the GIF with these changes?
- The GIF seems to be based on an old version of the frontend, since the preview caption and buttons are different. Update the frontend to be based on latest version of code
- We don't need to show the data upload cell, it would be good to just zoom in to the statement on where the LuxSQLTable is created. Something similar to this, where the import, pycopg2 connection, and LuxSQLTable declaration is separated into three cells:
import lux
import pandas as pdimport psycopg2
connection = psycopg2.connect("host=localhost dbname=testdb user=testuser password=testpass")
lux.config.set_SQL_connection(connection)tbl = lux.LuxSQLTable("college")
tbl
3) The GIF doesn't need to spend a lot of time scrolling through and clicking on the tabs of the widget, simply Toggle to the Lux view and clicking on one tab would be sufficient.
dorisjlee
approved these changes
Apr 17, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Update to executor documentation.