Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: Fix millisecond issue for omniscidb, mysql, postgresql and pandas #2170

Merged
merged 3 commits into from
Apr 9, 2020
Merged

BUG: Fix millisecond issue for omniscidb, mysql, postgresql and pandas #2170

merged 3 commits into from
Apr 9, 2020

Conversation

xmnlab
Copy link
Contributor

@xmnlab xmnlab commented Apr 1, 2020

Fix #2167
Fix #2166
Fix #2168
Fix #2169

@xmnlab xmnlab requested a review from jreback April 1, 2020 20:45
@xmnlab
Copy link
Contributor Author

xmnlab commented Apr 1, 2020

This PR is done for review. thanks!

@xmnlab xmnlab force-pushed the fix-millisecond-extraction-issue branch from 28e1fbf to 737850d Compare April 5, 2020 01:42
@datapythonista datapythonista added the bug Incorrect behavior inside of ibis label Apr 6, 2020
jreback
jreback suggested changes Apr 6, 2020
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks fine, ping when release note is updated and green.

docs/source/release.rst Outdated
@@ -13,6 +13,7 @@ Release Notes
* :feature:`2126` Add translation rules for isnull() and notnull() for pyspark backend
* :feature:`2062` Implement read_csv for omniscidb backend
* :feature:`2097` Date, DateDiff and TimestampDiff implementations for OmniSciDB
* :bug:`2170` Fix millisecond issue for OmniSciDB, MySQL, PostgreSQL and Pandas backends
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add all of the issues numbers here (same issue if fine)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added all the issues number near each backend name.

@xmnlab xmnlab force-pushed the fix-millisecond-extraction-issue branch from 737850d to 99a1682 Compare April 6, 2020 21:20
jreback
jreback reviewed Apr 6, 2020
View reviewed changes
docs/source/release.rst Outdated
@@ -13,6 +13,7 @@ Release Notes
* :feature:`2126` Add translation rules for isnull() and notnull() for pyspark backend
* :feature:`2062` Implement read_csv for omniscidb backend
* :feature:`2097` Date, DateDiff and TimestampDiff implementations for OmniSciDB
* :bug:`2170` Fix millisecond issue for OmniSciDB #2167, MySQL #2169, PostgreSQL #2166 and Pandas #2168 backends
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add then the same as :bug:`2170` otherwise these don't render

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done! I used :issue: for that. thanks!

@xmnlab xmnlab force-pushed the fix-millisecond-extraction-issue branch from 99a1682 to bc734d6 Compare April 7, 2020 14:51
@xmnlab
Copy link
Contributor Author

xmnlab commented Apr 7, 2020

test output:

11:04 $ IBIS_TEST_MYSQL_PORT=3307 pytest -vv -k millisecond ibis/tests/all/test_temporal.py::test_timestamp_extract
============================ test session starts =============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.0 -- /home/xmn/miniconda3/envs/ibis-dev/bin/python
cachedir: .pytest_cache
rootdir: /home/xmn/dev/quansight/ibis-project/ibis, inifile: setup.cfg
plugins: mock-2.0.0, cov-2.8.1, xdist-1.31.0, forked-1.1.2
collected 91 items / 78 deselected / 13 selected                             

ibis/tests/all/test_temporal.py::test_timestamp_extract[BigQuery-millisecond] SKIPPED [  7%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[Clickhouse-millisecond] XFAIL [ 15%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[Csv-millisecond] PASSED [ 23%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[HDF5-millisecond] SKIPPED [ 30%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[Impala-millisecond] PASSED [ 38%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[MySQL-millisecond] PASSED [ 46%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[OmniSciDB-millisecond] PASSED [ 53%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[Pandas-millisecond] PASSED [ 61%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[Parquet-millisecond] PASSED [ 69%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[PostgreSQL-millisecond] PASSED [ 76%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[PySpark-millisecond] XFAIL               [ 84%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[SQLite-millisecond] XFAIL                [ 92%]
ibis/tests/all/test_temporal.py::test_timestamp_extract[Spark-millisecond] XFAIL                 [100%]

================== 7 passed, 2 skipped, 78 deselected, 4 xfailed in 85.39s (0:01:25) ===================

xmnlab
xmnlab commented Apr 7, 2020
View reviewed changes
ibis/pandas/execution/temporal.py
@@ -33,15 +33,20 @@ def execute_extract_timestamp_field_timestamp(op, data, **kwargs):
return getattr(data, field_name)


@execute_node.register(ops.ExtractTemporalField, pd.Series)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

execute_extract_timestamp_field_series was moved up to be close to above function execute_extract_timestamp_field_timestamp

@jreback jreback added this to the Next Bugfix Release milestone Apr 9, 2020
@jreback jreback merged commit 6d26682 into ibis-project:master Apr 9, 2020
@jreback
Copy link
Contributor

jreback commented Apr 9, 2020

thanks @xmnlab

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

Assignees
No one assigned
Labels
bug Incorrect behavior inside of ibis
Projects
None yet
Milestone
No milestone
3 participants
@xmnlab @jreback @datapythonista