Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to executor documentation. #363

Merged
merged 244 commits into from
Apr 17, 2021
Merged
Show file tree
Hide file tree
Changes from 243 commits
Commits
Show all changes
244 commits
Select commit Hold shift + click to select a range
308d99c
Merging Recent SQL Executor changes
19thyneb Oct 15, 2020
daa9a0d
Fix to Validator
19thyneb Oct 16, 2020
804b0dc
Fix Bug with Widget Rendering
19thyneb Oct 17, 2020
676f3e0
Added Number of Observations to MetaData, Fixed Interestingness issue…
19thyneb Oct 19, 2020
8763df9
Re-added Licensing Headers
19thyneb Oct 19, 2020
c2b0b46
Adding Recent frame.py changes
19thyneb Oct 19, 2020
1b08461
Adjusted SQL Executor Tests
19thyneb Oct 19, 2020
38c5e7e
Update Frame with new Action Registering
19thyneb Oct 22, 2020
14d2f90
Resolving Conflicts in frame.py
19thyneb Oct 22, 2020
78d8e10
Merge branch 'sql-engine' into Database-Executor
thyneb19 Oct 22, 2020
d783b4c
Commenting out local SQL Executor tests
19thyneb Oct 22, 2020
c03e001
Merge branch 'Database-Executor' of https://github.com/thyneb19/lux i…
19thyneb Oct 22, 2020
8f0e643
Update correlation.py
dorisjlee Oct 22, 2020
d365d52
Update frame.py
dorisjlee Oct 22, 2020
7da2992
Fixing Code Format
19thyneb Oct 22, 2020
f1b7c8b
Cleaning up Pandas Executor imports
19thyneb Oct 22, 2020
d97f0e4
Fix Validation Bug
19thyneb Oct 22, 2020
582b370
Changed metadata variable name
19thyneb Oct 23, 2020
bc87eb8
Moving Current SQL Executor changes to new branch (#119)
thyneb19 Oct 23, 2020
554c71f
Merge remote-tracking branch 'upstream/sql-engine' into Database-Exec…
19thyneb Oct 23, 2020
d65687a
Added script to generate Postgresql database
19thyneb Oct 25, 2020
7243b2f
Update upload_car_data.py
19thyneb Oct 25, 2020
2add76f
Updated script name in travis.yml
19thyneb Oct 25, 2020
cf74beb
Removed unnecessary import from travis.yml
19thyneb Oct 25, 2020
14d52b8
Added psycopg2 to requirements.txt
19thyneb Oct 25, 2020
379517d
Creating Postgres test database in travis
19thyneb Oct 25, 2020
a72f236
Fixed directory issue
19thyneb Oct 25, 2020
4a9db88
Added test environment for Postgresql Executor (#124)
thyneb19 Oct 26, 2020
e947fd6
Updated SQL Executor Tests
19thyneb Oct 26, 2020
1234009
Added sql_executor example notebook, minor bug fix
19thyneb Oct 31, 2020
a5eff98
Cleaned SQL Executor Example Notebook
19thyneb Nov 2, 2020
5e18fd8
Update custom action reference to executor
19thyneb Nov 2, 2020
6890408
Added example notebook, fixed variable reference (#130)
thyneb19 Nov 3, 2020
8029ac1
Updated Tests, Added benchmarking for SQL Executor
19thyneb Nov 6, 2020
bca1f0c
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Nov 7, 2020
4046579
Merge with upstream branch, added preliminary benchmarking code
19thyneb Nov 7, 2020
907f215
Added 2D Binning functionality to SQL Executor
19thyneb Nov 15, 2020
2b24129
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Nov 19, 2020
21ba332
Updated 2D Binning Functionality
19thyneb Nov 19, 2020
b7ac30c
Added Heatmap generation to SQL Executor, Bug fix in PandasExecutor
19thyneb Nov 22, 2020
ad32db7
Updated Code Formatting with Black
19thyneb Nov 22, 2020
8948538
Merge branch 'sql-engine' into Database-Executor
thyneb19 Nov 22, 2020
6f032f2
Update Requirements to include psycopg2
19thyneb Nov 22, 2020
06dbf14
Merge branch 'Database-Executor' of https://github.com/thyneb19/lux i…
19thyneb Nov 22, 2020
aff131f
Update upload_car_data.py
19thyneb Nov 22, 2020
0ca0d98
Update Compiler tests to use correct test DB
19thyneb Nov 22, 2020
776489f
Removed Benchmarking Code
19thyneb Nov 22, 2020
7d00742
Fixing Black Formatting
19thyneb Nov 22, 2020
bb77423
Updating SQL-Engine branch to main branch, Adding Heatmap Functional…
thyneb19 Nov 25, 2020
7101df1
Moved Executor Parameters to Global Config
19thyneb Nov 25, 2020
4e3ecb7
Black formatting
19thyneb Nov 25, 2020
a48cdc7
Moved table_name parameter to frame.py. Removed executor_type parameter
19thyneb Nov 27, 2020
bccd61a
Fixed reference to table_name parameter
19thyneb Nov 28, 2020
5794718
Adjusted Functions to Set SQL Connection
19thyneb Nov 29, 2020
fb61c27
Merge branch 'master' into Database-Executor
19thyneb Nov 29, 2020
727315b
Update SQLExecutor name parameter
19thyneb Nov 29, 2020
52e6be4
Merging master branch with sql engine. Moving executor parameters to …
19thyneb Nov 29, 2020
2cc5b10
Merge branch 'master' into Database-Executor
19thyneb Nov 29, 2020
10e9ea1
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Nov 29, 2020
b3e1e88
Parameter Bug Fix
19thyneb Nov 29, 2020
d774a41
Reference Fix in Warning
19thyneb Nov 29, 2020
9747848
Black Formatting
19thyneb Nov 29, 2020
fbafb96
Merge branch 'sql-engine' into Database-Executor
19thyneb Nov 29, 2020
ecfb434
Black formatting
19thyneb Nov 29, 2020
47ece2b
Merge remote-tracking branch 'upstream/master'
19thyneb Nov 30, 2020
3e68db3
Fix Executor Reference
19thyneb Nov 30, 2020
3a5473e
Update frame.py
dorisjlee Dec 2, 2020
68d1600
Moved set functions to global config
19thyneb Dec 2, 2020
b6a5e99
Cleaned up executor imports, Fixed issue in AltairRenderer
19thyneb Dec 3, 2020
9a1748c
Black formatting
19thyneb Dec 3, 2020
77a390d
Merged changes from Master branch, Moved Executor Parameters to Confi…
thyneb19 Dec 9, 2020
cb22f5c
Merge remote-tracking branch 'upstream/master'
19thyneb Dec 20, 2020
5223a11
Fixed Index Issue in Pandas Executor
19thyneb Dec 20, 2020
ad89594
Added tests for set_index functions
19thyneb Dec 21, 2020
4718c05
Black formatting
19thyneb Dec 21, 2020
2a7d4b2
Merge remote-tracking branch 'upstream/master'
19thyneb Dec 22, 2020
269c062
Update Pandas Executor to handle NA values
19thyneb Dec 22, 2020
79b381a
Merge branch 'master' into Database-Executor
19thyneb Dec 27, 2020
fd93cd4
Update to Config, and Compiler/Interestingness Tests
19thyneb Dec 27, 2020
7324e29
Merge branch 'sql-engine' into Database-Executor
19thyneb Dec 27, 2020
674d828
Black formatting
19thyneb Dec 27, 2020
2931442
Update Requirements.txt
19thyneb Dec 27, 2020
2fa2dca
Update to Sql-Engine (#190)
thyneb19 Dec 28, 2020
7cc9626
Update SQL Executor Documentation
19thyneb Jan 4, 2021
1d517d8
Updated SQLExecutor Example Notebook
19thyneb Jan 4, 2021
f5a358c
Merge branch 'sql-engine' into Database-Executor
19thyneb Jan 4, 2021
3a27d76
Black Formatting
19thyneb Jan 4, 2021
d187b39
Update to SQL Executor Example Notebook (#193)
thyneb19 Jan 4, 2021
1229809
Update to SQL Executor Tests
19thyneb Jan 4, 2021
c405f67
Update Travis file and SQL Executor Tests
19thyneb Jan 5, 2021
7eb665e
Merge branch 'sql-engine' into Database-Executor
thyneb19 Jan 5, 2021
86a35fa
Update .travis.yml
19thyneb Jan 5, 2021
d539b83
Merge branch 'Database-Executor' of https://github.com/thyneb19/lux i…
19thyneb Jan 5, 2021
d3819b7
Merge pull request #196 from thyneb19/Database-Executor
thyneb19 Jan 5, 2021
06a8b9c
Merging changes
dj-khandelwal Jan 7, 2021
50a36e8
fixed merge conflict issues. vis.data shows None DF.
dj-khandelwal Jan 8, 2021
7479129
Merging changes
dj-khandelwal Jan 8, 2021
3966521
Merge master into sql-engine + minor mergeconflict fixes
dj-khandelwal Jan 8, 2021
6d1ad6c
Removing the PYNB
dj-khandelwal Jan 8, 2021
085f036
Cleaning up obsolete code
dj-khandelwal Jan 8, 2021
289f670
Merging the master branch changes into sql-engine (#208)
dj-khandelwal Jan 9, 2021
812a27e
Merging changes after the travis yml script fix
dj-khandelwal Jan 10, 2021
0ed2f0e
Merge conflicts resolved
dj-khandelwal Jan 10, 2021
af0e742
Updating sql-engine after merge with the travis build fix (#213)
dj-khandelwal Jan 10, 2021
96a3742
Merged in SQL-Engine changes, Cleaned up method to connect Lux to SQL…
19thyneb Jan 10, 2021
522c616
Merge pull request #214 from thyneb19/Database-Executor
thyneb19 Jan 10, 2021
973a93e
Fixed SQLExecutor's Variable Handling
19thyneb Jan 12, 2021
97d1281
Black Formatting
19thyneb Jan 19, 2021
395dfd6
Merge pull request #236 from thyneb19/Database-Executor
thyneb19 Jan 19, 2021
04e449f
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Jan 19, 2021
0defa44
Updated data_type reference in SQLExecutor
19thyneb Jan 19, 2021
02906f9
Update Datetime Numeric Check
19thyneb Jan 21, 2021
56ff766
Merge pull request #237 from thyneb19/Database-Executor
thyneb19 Jan 21, 2021
9a96c55
Update travis file to generate Postgres Test DB
19thyneb Jan 21, 2021
daf9c83
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Jan 21, 2021
5d9a999
Update test_vis.py
19thyneb Jan 21, 2021
105942a
Merge pull request #238 from thyneb19/Database-Executor
thyneb19 Jan 21, 2021
fabdf2e
Merging from lux/sql-engine
dj-khandelwal Jan 23, 2021
15f290e
Improved SQLExecutor Warning Handling, Bugfix with 2D Binning
19thyneb Feb 8, 2021
cfdbf16
Adjustment to AltairRenderer
19thyneb Feb 10, 2021
660655c
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Feb 10, 2021
04b43dc
Merge branch 'sql-engine' into Database-Executor
19thyneb Feb 10, 2021
1b8b99d
Black Formatting
19thyneb Feb 10, 2021
4fd9a00
Merge pull request #261 from thyneb19/Database-Executor
thyneb19 Feb 10, 2021
ca8065a
Merge branch 'sql-engine' of https://github.com/lux-org/lux into sql-…
dj-khandelwal Feb 11, 2021
8327fda
Added Better Null Value Handling to SQLExecutor
19thyneb Feb 15, 2021
a7e2220
SQLExecutor execute_binning Fixes
19thyneb Feb 18, 2021
4394ad6
Update SQLExecutor Tests
19thyneb Feb 18, 2021
d0ed14a
Black Formatting
19thyneb Feb 18, 2021
3e649c2
Merge branch 'sql-engine' into Database-Executor
19thyneb Feb 18, 2021
b985393
Merge pull request #267 from thyneb19/Database-Executor
thyneb19 Feb 18, 2021
6134ce5
Merge branch 'sql-engine' of https://github.com/lux-org/lux into sql-…
dj-khandelwal Feb 18, 2021
43e0e63
Removing Test Print Statement
19thyneb Feb 18, 2021
b3ee36a
Optimizing SQLExecutor 2D Binning
19thyneb Feb 20, 2021
de1e0a1
Black Formatting
19thyneb Feb 20, 2021
4572dd2
Merge branch 'sql-engine' into Database-Executor
19thyneb Feb 21, 2021
5b0cd2b
Merge pull request #283 from thyneb19/Database-Executor
thyneb19 Feb 21, 2021
c1ea16d
Merge branch 'sql-engine' of https://github.com/lux-org/lux into sql-…
dj-khandelwal Feb 22, 2021
c8dfa12
Added Null Value Filtering to SQLExecutor
19thyneb Feb 23, 2021
cfa67b7
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Feb 23, 2021
180556c
Fixed Handling of String Filter Values
19thyneb Feb 25, 2021
45d6c07
Added Better Handling for Lazy Execution
19thyneb Feb 26, 2021
46f441c
Merge pull request #288 from thyneb19/Database-Executor
thyneb19 Feb 26, 2021
06e0919
Merge branch 'sql-engine' of https://github.com/lux-org/lux into sql-…
dj-khandelwal Feb 26, 2021
4fa1534
Updated Heatmap Threshold
19thyneb Feb 26, 2021
18eb80d
Updated SQLExecutor to Not Include Null Values in Metadata
19thyneb Feb 27, 2021
4c19e4d
Black formatting
19thyneb Feb 27, 2021
84d824e
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Mar 2, 2021
d4ee7da
Merge pull request #292 from thyneb19/Database-Executor
thyneb19 Mar 2, 2021
7027d2c
Merge remote-tracking branch 'upstream/master' into Database-Executor
19thyneb Mar 6, 2021
650355f
Fixed Issue with SQLExecutor and Custom Actions
19thyneb Mar 6, 2021
a1dea61
Merge pull request #300 from thyneb19/Database-Executor
thyneb19 Mar 10, 2021
2dab7ed
Created LuxSQLTable Object
thyneb19 Mar 12, 2021
e36c8a1
Merge pull request #303 from thyneb19/Database-Executor
thyneb19 Mar 12, 2021
04591af
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Mar 13, 2021
fa917fb
Added query parameter to Vis objects
thyneb19 Mar 13, 2021
2876de9
Some minor datatype detection changes to SQLExecutor
dj-khandelwal Mar 13, 2021
df3ee00
Fresh pull merge
dj-khandelwal Mar 13, 2021
c3de6c6
Revert "Added query parameter to Vis objects"
thyneb19 Mar 14, 2021
ce2017b
Merge pull request #305 from thyneb19/Database-Executor
thyneb19 Mar 14, 2021
4019070
Update python-app.yml to set up Postgres in test instance
thyneb19 Mar 14, 2021
bca4e5e
Removed Example Notebooks for SQLExecutor
thyneb19 Mar 14, 2021
d19d2c2
Update to Script Uploading Car dataset to Postgres
thyneb19 Mar 14, 2021
a7134b0
Update python-app.yml
thyneb19 Mar 14, 2021
af0fe9d
Merge pull request #307 from thyneb19/Database-Executor
thyneb19 Mar 14, 2021
626cc5f
Update Lux SQLTable Frontend
thyneb19 Mar 16, 2021
0d1c557
Delete sql_benchmarking.csv
thyneb19 Mar 16, 2021
3a0fbd2
Update CONTRIBUTING.md
dorisjlee Mar 16, 2021
87fe7d7
Update CONTRIBUTING.md
dorisjlee Mar 16, 2021
916f73a
cleaning up PR
dorisjlee Mar 16, 2021
4ad6ec4
fix flights data upload
dorisjlee Mar 16, 2021
6fe8433
Some changes for length calculation
dj-khandelwal Mar 17, 2021
2fcfb8f
Merging changes after the update for the setup script for GitHub test…
dj-khandelwal Mar 17, 2021
b1b56d7
Test Commit of SQLTable
sophiahhuang Mar 17, 2021
be2d1a2
Cleaned Up Test Suite
thyneb19 Mar 17, 2021
e8f6106
Some changes to datatype, test SQL Executor, count queries to count(1…
dj-khandelwal Mar 18, 2021
e2e1482
removing travis.tml and date_utils import from sqlexecutor
dj-khandelwal Mar 18, 2021
a6bd2fe
Merge pull request #311 from dj-khandelwal/sql-engine
thyneb19 Mar 18, 2021
799f47d
Merge branch 'sql-engine' into Database-Executor
thyneb19 Mar 18, 2021
8f1e21f
Clean up LuxSQLTable
thyneb19 Mar 18, 2021
0e760c2
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Mar 18, 2021
7f5c1b8
Update test_vis.py
thyneb19 Mar 18, 2021
247387d
Merge pull request #312 from thyneb19/Database-Executor
thyneb19 Mar 18, 2021
460e6e2
Black Formatting
thyneb19 Mar 18, 2021
c8a9471
Merge pull request #313 from thyneb19/Database-Executor
thyneb19 Mar 18, 2021
34b5c45
Updated LuxSQLTable notification
sophiahhuang Mar 19, 2021
39d8f41
Merge branch 'sql-engine' into sql-engine
thyneb19 Mar 19, 2021
d68b16d
Remove Out of Date LuxSQLTableNotice
sophiahhuang Mar 19, 2021
f190a55
Black Reformatting
thyneb19 Mar 19, 2021
fcad97a
Merge pull request #314 from sophiahhuang/sql-engine
thyneb19 Mar 19, 2021
ae26397
Remove redundantly added parameters
thyneb19 Mar 19, 2021
562be13
Update config error handling and LuxSQLTable description
thyneb19 Mar 22, 2021
e6b2f29
Clean up Vis.py Add greater connection visibility in LuxSQLTable
thyneb19 Mar 23, 2021
d9073af
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Mar 23, 2021
922f2a7
Update executor.rst
thyneb19 Mar 23, 2021
7b67f06
Refactor length parameter to _length
thyneb19 Mar 25, 2021
0323be0
Update test_interestingness.py
thyneb19 Mar 25, 2021
bea42cc
Added _length Parameter to LuxSQLTable
thyneb19 Mar 25, 2021
e9afa92
Black Reformatting, Reverting_length change in LuxDataFrame
thyneb19 Mar 25, 2021
d296c9b
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Mar 26, 2021
7c7dcd3
Update LuxSQLTable __len__() and metadata computation
thyneb19 Mar 26, 2021
8d6cf4b
Removed unnecessary __repr__() function
thyneb19 Mar 26, 2021
e350ab4
Updated LuxSQLTable repr
thyneb19 Mar 26, 2021
5d1a2f4
Revert "Updated LuxSQLTable repr"
thyneb19 Mar 27, 2021
48c1b57
Revert "Revert "Updated LuxSQLTable repr""
thyneb19 Mar 27, 2021
b5998c7
Revert "Update LuxSQLTable __len__() and metadata computation"
thyneb19 Mar 27, 2021
75c5cae
Merge pull request #327 from thyneb19/Database-Executor
thyneb19 Mar 27, 2021
6f597c2
Revert "Revert "Update LuxSQLTable __len__() and metadata computation""
thyneb19 Mar 27, 2021
7999ad6
Cleaned up datatype and SQLExecutor checks
thyneb19 Apr 3, 2021
db736d1
Update LuxSQLTable __len__() and metadata computation"" (#331)
thyneb19 Apr 7, 2021
e92dbd6
Merge branch 'sql-engine' into Database-Executor
thyneb19 Apr 7, 2021
5399097
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Apr 7, 2021
2d24a3b
Merge pull request #347 from thyneb19/Database-Executor
thyneb19 Apr 8, 2021
6922d3b
Black Reformatting
thyneb19 Apr 8, 2021
bf3cb0f
Merge pull request #348 from thyneb19/Database-Executor
thyneb19 Apr 8, 2021
694c7d7
Update executor.rst
NiStannum Apr 9, 2021
40b85b1
minor changes to requirements and cleanup
dorisjlee Apr 11, 2021
2298f13
Merge branch 'master' into sql-engine
dorisjlee Apr 11, 2021
801f3cd
Removed psycopg2 from Lux requirements
thyneb19 Apr 11, 2021
284f5ba
Merge branch 'sql-engine' into Database-Executor
thyneb19 Apr 11, 2021
1e02ad6
Merge pull request #352 from thyneb19/Database-Executor
thyneb19 Apr 11, 2021
68c7747
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Apr 11, 2021
f11e772
Revert "Merge remote-tracking branch 'upstream/master' into Database-…
thyneb19 Apr 11, 2021
a309361
Merge branch 'Database-Executor' of https://github.com/thyneb19/lux i…
thyneb19 Apr 11, 2021
1632005
Merge pull request #353 from thyneb19/Database-Executor
thyneb19 Apr 11, 2021
c94b5a6
add back merged overridden changes
dorisjlee Apr 11, 2021
50f8562
merge conflict fixed
dorisjlee Apr 11, 2021
f1268ef
Separate Pandas and SQL test suite
thyneb19 Apr 12, 2021
edaad70
Separate Pandas and SQL test suite (#354)
thyneb19 Apr 12, 2021
549c1fd
Merge remote-tracking branch 'upstream/master' into Database-Executor
thyneb19 Apr 12, 2021
5600bbd
Merge branch 'sql-engine' into Database-Executor
thyneb19 Apr 12, 2021
5f1d10c
Merge pull request #355 from thyneb19/Database-Executor
thyneb19 Apr 12, 2021
94f9aa1
Black reformatting
thyneb19 Apr 12, 2021
8da2a6b
Update python-app.yml
dorisjlee Apr 12, 2021
aa66626
Fixing issues in test suite
thyneb19 Apr 12, 2021
51133a3
Fixing Lux SQL test suite
thyneb19 Apr 12, 2021
a6cef02
Added documentation for SQL Executor.
NiStannum Apr 15, 2021
1933463
Merge pull request #361 from NiStannum/sql-engine
thyneb19 Apr 15, 2021
20619e4
Update gif link in Executor documentation
thyneb19 Apr 16, 2021
d04b8fc
writing changes to documentation
dorisjlee Apr 16, 2021
34c2276
Update executor.rst
thyneb19 Apr 16, 2021
a6d524c
changes to docs
dorisjlee Apr 17, 2021
3d7b447
merge conflict resolved
dorisjlee Apr 17, 2021
3b84282
typo
dorisjlee Apr 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,14 +39,16 @@ This website contains pages that overview of the basic and advanced functionalit
:maxdepth: 1
:caption: Advanced Topics

source/advanced/map
source/advanced/datatype
source/advanced/date
source/advanced/map
source/advanced/indexgroup
source/advanced/custom
source/advanced/architecture
source/advanced/interestingness
source/advanced/executor
source/advanced/datatype
source/advanced/interestingness



.. toctree::
:maxdepth: 1
Expand Down
6 changes: 3 additions & 3 deletions doc/source/advanced/date.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
********************************
Working with Dates
********************************
***************************************
Working with Temporal Datetime Columns
***************************************

.. note:: You can follow along this tutorial in a Jupyter notebook. [`Github <https://github.com/lux-org/lux-binder/blob/master/tutorial/tutorial/5-datetime.ipynb>`_] [`Binder <https://mybinder.org/v2/gh/lux-org/lux-binder/master?urlpath=tree/tutorial/5-datetime.ipynb>`_]

Expand Down
61 changes: 42 additions & 19 deletions doc/source/advanced/executor.rst
Original file line number Diff line number Diff line change
@@ -1,47 +1,66 @@
****************
Execution Engine
****************

Fetching the data required for generating visualizations can be computationally expensive, especially on large datasets. Lux provides a extensible framework for users to pick their own execution backend for data processing. We currently support Pandas (default, :mod:`lux.executor.PandasExecutor`) and SQL (:mod:`lux.executor.SQLExecutor`). In this tutorial, we explain how to use switch to SQL as an execution backend, as an example of how you can use a different data processing mechanism in Lux.

Please refer to :mod:`lux.executor.Executor`, if you are interested in extending Lux for your own execution backend.
**************************
Working with SQL Databases
**************************

Lux provides an extensible framework for users to pick their own execution backend for data processing. We currently support Pandas (:mod:`lux.executor.PandasExecutor`) and SQL (:mod:`lux.executor.SQLExecutor`) as the execution engine. By default, Lux leverages Pandas as its execution backend; in other words, the data processing code is performed as a set of Pandas operations on top of dataframe. In this tutorial, we further explain how Lux can be used with SQL with tables inside a Postgres database.

.. note:: You can follow a tutorial describing how Lux can be used with data inside a Postgres database in a Jupyter notebook. [`Github <https://github.com/lux-org/lux-binder-sql/blob/master/notebooks/Using%20Lux%20with%20SQL%20Databases.ipynb>`_] [`Binder <https://mybinder.org/v2/gh/lux-org/lux-binder-sql/HEAD>`_]

SQL Executor
=============
thyneb19 marked this conversation as resolved.
Show resolved Hide resolved

Lux extends its visualization exploration operations to data within SQL databases. By using the SQL Executor, users can specify a SQL database to connect a LuxSQLTable for generating all the visualizations recommended in Lux.
Lux extends its visualization capabilities to SQL within Postgres databases. By using the SQLExecutor, users can create a :code:`LuxSQLTable` that connects to a Postgres database. When the :code:`LuxSQLTable` object is printed out, Lux displays a subset of the data and recommends a default set of visualizations to display.

.. image:: https://github.com/lux-org/lux-resources/blob/master/doc_img/SQLexecutor1.gif?raw=true
:width: 900
:align: center


What is the SQL Executor?
==========================

It is common for data to be stored within a relational database, such as Postgres.
The execution engine in Lux processes the data in order to generate the data required for the visualization. By default, Lux uses Pandas as its execution engine.
However, fetching the data required for generating visualizations can be computationally expensive. Database users may not be able to pull in the entire dataset, either due to a lack of permissions or due to the data being too large to work with on a local machine. Thus, in order to leverage Lux's capabilities, you can use the :code:`LuxSQLTable` to work with data stored inside a Postgres database. A :code:`LuxSQLTable` represents a SQL table with the Postgres database. The :code:`LuxSQLTable` contains a skeleton of the dataframe schema and does not store the entire data in the database. (Underneath the hoods, :code:`LuxSQLTable` is a database that serve as the LuxDataFrame for a table. However, note that since :code:`LuxSQLTable` is not a dataframe, you cannot use the usual Pandas Dataframe functions on :code:`LuxSQLTable`.)

Connecting Lux to a Database
----------------------------
thyneb19 marked this conversation as resolved.
Show resolved Hide resolved

Before Lux can operate on data within a Postgresql database, users have to connect their LuxSQLTable to their database.
To do this, users first need to specify a connection to their SQL database. This can be done using the psycopg2 package's functionality.
.. note:: To run these examples with your own Postgresql database locally, please follow `these instructions <https://github.com/lux-org/lux-binder-sql/blob/master/notebooks/Using%20Lux%20with%20SQL%20Databases.ipynb>`_ how to set up and populate the appropriate example database and table.

Before Lux can operate on data within a Postgres database, users have to connect their LuxSQLTable to their database.
To do this, users first need to specify a connection to their SQL database. This can be done using `psycopg2 <https://pypi.org/project/psycopg2/>`_ or `sqlalchemy <https://www.sqlalchemy.org/>`_ SQL database connectors, shown as follows:

.. code-block:: python

import psycopg2
connection = psycopg2.connect("dbname=example_database user=example_user, password=example_password")
connection = psycopg2.connect("dbname=postgres_db_name user=example_user password=example_user_password")

thyneb19 marked this conversation as resolved.
Show resolved Hide resolved
Once this connection is created, users can connect the lux config to the database using the set_SQL_connection command.
.. code-block:: python

from sqlalchemy import create_engine
engine = create_engine("postgresql://postgres:lux@localhost:5432")

Note that users will have to install these packages on their own if they want to connect Lux to their databases.
Once this connection is created, users can connect the lux config to the database using the :code:`set_SQL_connection` command.

.. code-block:: python

lux.config.set_SQL_connection(connection)

When the set_SQL_connection function is called, Lux will then populate the LuxSQLTable with all the metadata it needs to run its intent from the database table.
After the SQL connection is set, Lux fetches the details required to connect to your PostgreSQL database and generate useful recommendations.

Connecting a LuxSQLTable to a Table/View
--------------------------
----------------------------------------

LuxSQLTables can be connected to individual tables or views created within your Postgresql database. This can be done by either specifying the table/view name in the constructor.
LuxSQLTables can be connected to individual tables or views created within your Postgresql database. This can be done by specifying the table or view name in the constructor.
.. We are actively working on supporting joins between multiple tables. But as of now, the functionality is limited to one table or view per LuxSQLTable object only.

.. code-block:: python

sql_tbl = LuxSQLTable(table_name = "my_table")

You can also connect a LuxSQLTable to a table/view by using the set_SQL_table function.
Alternatively, you can also connect a LuxSQLTable to a table or view by using :code:`set_SQL_table`:

.. code-block:: python

Expand All @@ -51,17 +70,21 @@ You can also connect a LuxSQLTable to a table/view by using the set_SQL_table fu
Choosing an Executor
--------------------------


Once a user has created a connection to their Postgresql database, they need to change Lux's execution engine so that the system can collect and process the data properly.
By default Lux uses the Pandas executor to process local data in the LuxDataframe, but users will use the SQL executor when their LuxSQLTable is connected to a database.
By default, Lux uses the Pandas executor to process local data in the LuxDataframe, but users will use the SQL executor when their LuxSQLTable is connected to a database.
Users can specify the executor that Lux will use via the set_executor_type function as follows:

.. code-block:: python

lux_df.set_executor_type("SQL")

Once a LuxSQLTable has been connected to a Postgresql table and set to use the SQL Executor, users can take full advantage of Lux's visual exploration capabilities as-is. Users can set their intent to specify which variables they are most interested in and discover insightful visualizations from their database.
Once a LuxSQLTable has been connected to a Postgresql table and set to use the SQL Executor, users can take full advantage of Lux's visual exploration capabilities as-is to discover insightful visualizations from their database.

SQL Executor Limitations
--------------------------

While users can make full use of Lux's functionalities on data within a database table, they will not be able to use any of Pandas' Dataframe functions to manipulate the data in the LuxSQLTable object. Since the Lux SQL Executor delegates most data processing to the Postgresql database, it does not pull in the entire dataset into the Lux Dataframe. As such there is no actual data within the LuxSQLTable to manipulate, only the relevant metadata required to for Lux to manage its intent. Thus, if users are interested in manipulating or querying their data, this needs to be done through SQL or an alternative RDBMS interface.
While users can make full use of Lux's functionalities on data within a database table, they will not be able to use any of Pandas' Dataframe functions to manipulate the data in the LuxSQLTable object. Since the Lux SQL Executor delegates most data processing to the Postgresql database, it does not pull in the entire dataset into the Lux Dataframe. As such there is no actual data within the LuxSQLTable to manipulate, only the relevant metadata required for Lux to manage its intent. Thus, if users are interested in manipulating or querying their data, this needs to be done through SQL or an alternative RDBMS interface.

Currently, Lux's SQLExecutor does not support JOIN operation on SQL tables. Therefore, you cannot explore data and create recommended visualizations across multiple SQL tables only through Lux. We are consistently working on expanding the SQL capabilities of Lux, please let us know about how you're using the SQLExecutor and how we can improve the functionality `here <https://github.com/lux-org/lux/issues>`_ !

6 changes: 3 additions & 3 deletions doc/source/advanced/map.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
********************************
Working with Geographic Data
********************************
************************************
Working with Geographic Data Columns
************************************

This tutorial describes how geographic attributes can be visualized automatically with Lux.
Lux recognizes any columns named :code:`state` and :code:`country` that contains US States or worldwide countries as geographic attributes.
Expand Down
4 changes: 2 additions & 2 deletions doc/source/reference/gen/lux.vis.Vis.Vis.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
lux.vis.Vis.Vis
lux.vis.Vis.Vis
===============

.. currentmodule:: lux.vis.Vis
Expand All @@ -24,9 +24,9 @@ lux.vis.Vis.Vis
~Vis.remove_filter_from_spec
~Vis.set_intent
~Vis.to_altair
~Vis.to_vegalite
~Vis.to_code
~Vis.to_matplotlib
~Vis.to_vegalite



Expand Down