Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First version of omnisci window op #1771

Merged
merged 5 commits into from May 16, 2019

Conversation

Projects
2 participants
@xmnlab
Copy link
Collaborator

commented May 10, 2019

This PR adds window operations for OmniSci/MapD backend:

  • lag
  • firstvalue
  • lead
  • lastvalue
  • mean
  • min
  • max
  • sum

@xmnlab xmnlab referenced this pull request May 10, 2019

Open

Ibis: Current Window Function Support #33

8 of 15 tasks complete

@xmnlab xmnlab marked this pull request as ready for review May 11, 2019

@xmnlab xmnlab requested a review from cpcloud May 11, 2019

@cpcloud cpcloud added this to the Next Feature Release milestone May 11, 2019

@cpcloud cpcloud added this to In progress in OmniSci via automation May 11, 2019

@cpcloud

This comment has been minimized.

Copy link
Member

commented May 11, 2019

@xmnlab Can you split this into 2 PRs? One with all the build/data loading related changes and one with the window changes?

@cpcloud

This comment has been minimized.

Copy link
Member

commented May 12, 2019

@xmnlab Can you rebase here? Sorry for the merge conflicts, let me know if you'd like me to help resolve them.

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 13, 2019

@cpcloud ! sounds pretty good .. I will open a new PR with the changes related to new omnisci database docker. and after that I will move this PR forward :)

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 13, 2019

the PR for the omnisci ci: updating #1781

@xmnlab xmnlab force-pushed the Quansight:add-mapd-window-support branch from 4626e52 to e8da5fc May 14, 2019

@codecov

This comment has been minimized.

Copy link

commented May 14, 2019

Codecov Report

Merging #1771 into master will decrease coverage by 2.01%.
The diff coverage is 51.11%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1771      +/-   ##
==========================================
- Coverage   87.57%   85.56%   -2.02%     
==========================================
  Files          80       80              
  Lines       15145    15190      +45     
  Branches     1958     1969      +11     
==========================================
- Hits        13263    12997     -266     
- Misses       1529     1834     +305     
- Partials      353      359       +6
Impacted Files Coverage Δ
ibis/mapd/operations.py 71.32% <51.11%> (-2.51%) ⬇️
ibis/bigquery/client.py 41.1% <0%> (-53.39%) ⬇️
ibis/bigquery/compiler.py 59.92% <0%> (-37.5%) ⬇️
ibis/bigquery/udf/api.py 80.48% <0%> (-14.64%) ⬇️
ibis/bigquery/api.py 63.33% <0%> (-10.01%) ⬇️
ibis/impala/compiler.py 91.28% <0%> (-5.6%) ⬇️
ibis/pandas/client.py 85.63% <0%> (-3.45%) ⬇️
ibis/expr/schema.py 89.77% <0%> (-1.14%) ⬇️
ibis/expr/window.py 86.46% <0%> (-0.76%) ⬇️
ibis/expr/api.py 93.53% <0%> (-0.4%) ⬇️
... and 3 more
@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 14, 2019

@cpcloud it seems it is done for a review :)

Show resolved Hide resolved ibis/tests/all/test_window.py Outdated
Show resolved Hide resolved ibis/tests/all/test_window.py Outdated

OmniSci automation moved this from In progress to Needs review May 15, 2019

@xmnlab xmnlab force-pushed the Quansight:add-mapd-window-support branch from 4882410 to 462c06b May 15, 2019

xmnlab added some commits May 10, 2019

First version of omnisci window op; improving window ops tests
Move back some small changes

Fixed issue on omnisci config file

Fixed small issues; added protocol parameter

applying changes from upstream

@xmnlab xmnlab force-pushed the Quansight:add-mapd-window-support branch from 462c06b to ae38fa0 May 15, 2019

@cpcloud

This comment has been minimized.

Copy link
Member

commented May 15, 2019

Merging on green.

@cpcloud cpcloud self-assigned this May 15, 2019

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 15, 2019

thanks @cpcloud !

@cpcloud

This comment has been minimized.

Copy link
Member

commented May 16, 2019

Just fixed up the merge conflict in the github UI, will wait for at least one the py36/py37 builds to turn green then will merge. Thanks @xmnlab!

@cpcloud

This comment has been minimized.

Copy link
Member

commented May 16, 2019

@xmnlab Can you disable the any and all tests for mapd for now? I changed them to enable testing without calling into the cumulative operations interface to be more consistent with the rest of the test cases in that file.

Sorry for the churn!

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 16, 2019

@cpcloud sure .. I will do it now. thanks!

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 16, 2019

not sure why it broke just for py35 with this error:

INTERNALERROR> AssertionError: ('ibis/tests/all/test_window.py::test_window[MapD-cumany]', <WorkerController gw1>)
INTERNALERROR> assert not 'ibis/tests/all/test_window.py::test_window[MapD-cumany]'

maybe I should just add these ops as unsupported for window operations

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 16, 2019

@cpcloud it is done :)

@cpcloud

This comment has been minimized.

Copy link
Member

commented May 16, 2019

Thanks, I will merge when I get to the office around 10a Eastern!

@xmnlab

This comment has been minimized.

Copy link
Collaborator Author

commented May 16, 2019

@cpcloud cpcloud merged commit c2323b8 into ibis-project:master May 16, 2019

13 of 15 checks passed

codecov/patch 51.11% of diff hit (target 87.57%)
Details
codecov/project 85.56% (-2.02%) compared to 063035c
Details
ci/circleci: python35_test Your tests passed on CircleCI!
Details
ci/circleci: python36_benchmark Your tests passed on CircleCI!
Details
ci/circleci: python36_conda_build Your tests passed on CircleCI!
Details
ci/circleci: python36_docs Your tests passed on CircleCI!
Details
ci/circleci: python36_test Your tests passed on CircleCI!
Details
ci/circleci: python37_conda_build Your tests passed on CircleCI!
Details
ci/circleci: python37_test Your tests passed on CircleCI!
Details
ibis-project.ibis Build #20190516.4 succeeded
Details
ibis-project.ibis (WindowsCondaBuild py36) WindowsCondaBuild py36 succeeded
Details
ibis-project.ibis (WindowsCondaBuild py37) WindowsCondaBuild py37 succeeded
Details
ibis-project.ibis (WindowsTest py35) WindowsTest py35 succeeded
Details
ibis-project.ibis (WindowsTest py36) WindowsTest py36 succeeded
Details
ibis-project.ibis (WindowsTest py37) WindowsTest py37 succeeded
Details

OmniSci automation moved this from Needs review to Done May 16, 2019

@xmnlab xmnlab deleted the Quansight:add-mapd-window-support branch May 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.