FEAT: [mapd] Added float32 and geospatial types for create table from…
… schema resolves #1753 Author: Ivan Ogasawara <ivan.ogasawara@gmail.com> Closes #1755 from xmnlab/add-float32-sql-type-names and squashes the following commits: cea890e [Ivan Ogasawara] Applied suggestion from review 28bf106 [Ivan Ogasawara] Applied suggestion from review 435b525 [Ivan Ogasawara] Applied changes from review. 5316f93 [Ivan Ogasawara] [mapd] Added float32 and geospatial types for create table from schema
BUG: Fix preceding and following with None
Before this PR, calls like ```python ibis.window(preceding=(None, 1)) # ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING ibis.window(following=(1, None)) # ROWS BETWEEN 1 FOLLOWING AND UNBOUNDED FOLLOWING ``` would fail because of incorrect usage of comparison operators, comparing ``None`` to another value that is not ``None`` raising a ``TypeError``. This PR addresses the problem. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1765 from cpcloud/fix-prec-foll-unbounded and squashes the following commits: 0c53272 [Phillip Cloud] Disable mysql for now a9ce03f [Phillip Cloud] BUG: Fix preceding and following with None
Fixes DOC: conda create command listed no longer works
Fixes error in documentation, issue #1774 The following command in `developer.rst` no longer works for two reasons. One, the YAML file no longer exists. Two, creating a conda environment from YAML needs to be done with `conda env create` instead of `conda create`. Failing instruction: `conda create -n ibis36 -c conda-forge --file=ci/requirements-dev.yml python=3.6` Should now be: `conda env create -n ibis36 -f ci/requirements-3.6-dev.yml` Author: Scott Hajek <shajek@pivotal.io> Closes #1775 from scottcode/docs_developer and squashes the following commits: dbc353f [Scott Hajek] The following command in `developer.rst` no longer works for two reasons. One, the YAML file no longer exists. Two, creating a conda environment from YAML needs to be done with `conda env create` instead of `conda create`.
ENH/CLN: Enable window function tests where possible
Author: Phillip Cloud <cpcloud@gmail.com> Closes #1777 from cpcloud/fix-window-test and squashes the following commits: 50f3214 [Phillip Cloud] Zero based offset for ranking 5b28f09 [Phillip Cloud] Move line to avoid unnecessary pandas calls 0358c71 [Phillip Cloud] Rank by id 24e18bd [Phillip Cloud] Rank should always return an integer 25190a7 [Phillip Cloud] Fix the impala compiler to not generate frame clauses for certain operations 130e2bb [Phillip Cloud] ENH/CLN: Enable window function tests where possible
BUG: NullIf should enforce that its arguments are castable to a commo…
This PR makes all `XPASS`ing tests pass in the typical way, by customizing the behavior of two fixtures `xfail_backends` (all listed backends are permitted to fail) and `xpass_backends` (all backends *not* listed are permitted to fail) I've also added `xfail_strict = true` to `setup.cfg` to make sure that we don't introduce any `XPASS`ing tests in the future. Of course, that's always up for debate in the future but I think there's probably a way to customize `xfail`s similar to what I've done here such that we don't need to have any future `XPASS`es. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1779 from cpcloud/cleanup-xpass-tests and squashes the following commits: 5290185 [Phillip Cloud] Clean up most xpassing tests
ENH: Rewrite pandas execution to use topological sort instead of recu…
…rsion cc @toryhaavik for review Author: Phillip Cloud <cpcloud@gmail.com> Closes #1758 from cpcloud/pandas-toposort and squashes the following commits: a3292ea [Phillip Cloud] Finish out comment 1508a3b [Phillip Cloud] Remove stale comment 4363b1a [Phillip Cloud] Remove unnecessary parens 59fdde8 [Phillip Cloud] Ignore any and all rewrites in the postgres backend 2ff7432 [Phillip Cloud] Impala does not have an logical reduction API 686d1ed [Phillip Cloud] Use native bigquery logical reductions intead of bogus rewrite hacks bffcbbf [Phillip Cloud] Remove weird decorator hacks b336901 [Phillip Cloud] Handle the case where our computed column name is the same as one in our table 6e4dce6 [Phillip Cloud] Special case cumulative operations f80c853 [Phillip Cloud] Use any and all instead of cumany cumall d083c72 [Phillip Cloud] Handle cases with any and all where they do not have rolling or expanding dispatches 7d87c67 [Phillip Cloud] Handle any and all as reductions 616b038 [Phillip Cloud] Use topological sort in the pandas backend
First version of omnisci window op
This PR adds window operations for OmniSci/MapD backend: - [x] lag - [x] firstvalue - [x] lead - [x] lastvalue - [x] mean - [x] min - [x] max - [x] count - [x] sum Author: Ivan Ogasawara <ivan.ogasawara@gmail.com> Author: Phillip Cloud <cpcloud@gmail.com> Closes #1771 from xmnlab/add-mapd-window-support and squashes the following commits: ea8cefc [Ivan Ogasawara] Added win all and win any as unsupported 4377c61 [Ivan Ogasawara] Disable MapD for win all and any tests 824ed63 [Phillip Cloud] Merge branch 'master' into add-mapd-window-support ae38fa0 [Ivan Ogasawara] Added some xfails 997d7d1 [Ivan Ogasawara] First version of omnisci window op; improving window ops tests
Fixes outer_join generating LEFT join instead of full outer
Fixes issue #1772 Making new pull request from new branch freshly rebased on upstream master. Supercedes pull request #1773. Author: Scott Hajek <shajek@pivotal.io> Closes #1788 from scottcode/fix_full_outer_join2 and squashes the following commits: 4e2fdc4 [Scott Hajek] Shorten long source lines that were failing linter 3a6830d [Scott Hajek] Test for Alchemy-based full outer join 6c91ee9 [Scott Hajek] Fixes bug: `outer_join` generates LEFT join instead of full outer
Add min periods parameter to Window
We want to set min_periods=1 on rolling windows to avoid defaulting to min_periods equal to the window size n, thereby eliminating the necessity of NaNs in the first n-1 rows for trailing window operations. Author: Emily Reff <emily.reff@twosigma.com> Closes #1792 from emilyreff7/rolling and squashes the following commits: 267a8e2 [Emily Reff] Merge branch 'rolling' of https://github.com/emilyreff7/ibis into rolling 8efca1e [Emily Reff] Merge branch 'rolling' of https://github.com/emilyreff7/ibis into rolling 20f75d3 [Emily Reff] Merge branch 'rolling' of https://github.com/emilyreff7/ibis into rolling bb904ff [Emily Reff] adding min period parameter to Window constructor 1def76f [Emily Reff] adding min period parameter to Window constructor 2f124ee [Emily Reff] Merge branch 'master' of https://github.com/ibis-project/ibis into rolling 76700d7 [Emily Reff] Added lol
BUG/CI/BLD: Fix Python 3.5 dependency versions
@ian-r-rose Can you take a look at this and make sure that it matches our discussion re OmniSci on #1796? Author: Phillip Cloud <cpcloud@gmail.com> Closes #1798 from cpcloud/fix-py35-deps and squashes the following commits: 2bace3f [Phillip Cloud] Add pydata-google-auth to pip deps in conda env requirements 4216d7a [Phillip Cloud] Fix Python 3.5 dependency versions
`pymapd` recently gained the ability to connect via a pre- authenticated session ID. It would be nice if the Ibis mapd connector could do that as well. cc @xmnlab Author: Ian Rose <ian.r.rose@gmail.com> Closes #1796 from ian-r-rose/mapd-session-id and squashes the following commits: 05307e8 [Ian Rose] Use exception subclass for pymapd version error. cf2dd8c [Ian Rose] Use different session -- reconnecting seems to invalidate old pymapd session. dc80ca5 [Ian Rose] Do some version checking in the MapD client. 7f27d47 [Ian Rose] More snake_case fixes. c60e105 [Ian Rose] Skip session id test on python 3.5, since we can't install pymapd 0.12 there. 403cbef [Ian Rose] Use snake_case for session_id, simplify test assertion. 7e86e04 [Ian Rose] Fix arg name. 9715446 [Ian Rose] Add test for connecting via sessionid. 926f587 [Ian Rose] Allow session id to be passed into ibis mapd connection.
BUG: Allow projections from joins to attempt fusion
Closes #1783 cc @toryhaavik Author: Phillip Cloud <cpcloud@gmail.com> Closes #1797 from cpcloud/gh-1783-fix-select-on-join and squashes the following commits: 08dfac2 [Phillip Cloud] BUG: Allow projections from joins to attempt fusion
BUG: Chained mutate operations give wrong results
Closes #1799 This was kind of a nasty one. The query flattening optimization was far too aggressive. @toryhaavik Do you feel comfortable reviewing this? One downside to this fix is that simple, single expression columns are not flattened into their subquery anymore. This will be addressed in the future after the compiler is refactored. Author: Phillip Cloud <cpcloud@gmail.com> Closes #1812 from cpcloud/mutate-chain-bug and squashes the following commits: 31dedd6 [Phillip Cloud] BUG: Chained mutate operations give wrong results
consolidate trailing window functions
This constitutes one step of several required to extend Ibis trailing window functionality, where trailing_window will become more extensible to support various types of 'preceding' parameters. In this first step, trailing_window is able to handle not only integers but ibis intervals as well, using the type of the 'preceding' parameter to determine the 'how' of the window. Author: Emily Reff <emily.reff@twosigma.com> Closes #1809 from emilyreff7/window and squashes the following commits: b09bf4f [Emily Reff] validate preceding type is supported 6351c94 [Emily Reff] add unit test for util fn fe24c33 [Emily Reff] Merge branch 'master' of https://github.com/ibis-project/ibis into window 12ddb39 [Emily Reff] consolidate trailing window functions
PKG: Add pre-commit, black and isort to setup.py
This PR moves some packages that were previously being installed with pip to be installed by conda. It also adds pre-commit, black and isort to `setup.py`. Closes #1814 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1816 from cpcloud/update-envs and squashes the following commits: f95ca9f [Phillip Cloud] PKG: Add pre-commit, black and isort to setup.py
BUG: Do not give a frame clause where not necessary in window functions
Closes #1757 Author: Phillip Cloud <cpcloud@gmail.com> Closes #1817 from cpcloud/remove-frame-clause-from-alchemy and squashes the following commits: 1fded47 [Phillip Cloud] Remove custom Over implementation and require sqlalchemy>=1.1 5e642bc [Phillip Cloud] Proper links to pypi and codecov 28767bb [Phillip Cloud] BUG: Do not give a frame clause where not necessary in window functions
