New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid datetime string conversion giving SystemError with Python 3.6.0b3 #14561

Closed
jorisvandenbossche opened this Issue Nov 2, 2016 · 4 comments

Comments

Projects
None yet
4 participants
@jorisvandenbossche
Member

jorisvandenbossche commented Nov 2, 2016

From #14552, using Python 3.6.0b3 and pandas 0.19.0:

In [12]: pd.Timestamp('invalid')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
ValueError: Error parsing datetime string "invalid" at position 0 

The above exception was the direct cause of the following exception:

SystemError                               Traceback (most recent call last)
<ipython-input-12-91abde127db1> in <module>()
----> 1 pd.Timestamp('invalid')
pandas/tslib.pyx in pandas.tslib.Timestamp.__new__ (pandas/tslib.c:9932)()
pandas/tslib.pyx in pandas.tslib.convert_to_tsobject (pandas/tslib.c:25231)()
pandas/tslib.pyx in pandas.tslib.convert_str_to_tsobject (pandas/tslib.c:26851)()
pandas/src/datetime.pxd in datetime._string_to_dts (pandas/tslib.c:87106)()
SystemError: <class 'str'> returned a result with an error set

gives a SystemError (which gives problems in the indexing code etc), while with python < 3.6, you get a ValueError:

In [56]: pd.Timestamp('next')
...
series of "During handling of the above exception, another exception occurred:"
...

/home/joris/scipy/pandas/pandas/tslib.pyx in pandas.tslib.convert_str_to_tsobject (pandas/tslib.c:27281)()
   1537                     ts, dayfirst=dayfirst, yearfirst=yearfirst)
   1538             except Exception:
-> 1539                 raise ValueError("could not convert string to Timestamp")
   1540 
   1541     return convert_to_tsobject(ts, tz, unit, dayfirst, yearfirst)

ValueError: could not convert string to Timestamp
@stonebig

This comment has been minimized.

Show comment
Hide comment
@stonebig

stonebig Nov 7, 2016

Contributor

is it an abnormal change that we should report ? to numpy ? to python.org ? to ... ?

Contributor

stonebig commented Nov 7, 2016

is it an abnormal change that we should report ? to numpy ? to python.org ? to ... ?

@jorisvandenbossche

This comment has been minimized.

Show comment
Hide comment
@jorisvandenbossche

jorisvandenbossche Nov 7, 2016

Member

Possibly, but that depends on the underlying reason. The error occurs in the _string_to_dts C function, but I am not familiar enough with that code to be able to directly pinpoint the reason.

So it someone want to take a deeper look at this, very welcome!

Member

jorisvandenbossche commented Nov 7, 2016

Possibly, but that depends on the underlying reason. The error occurs in the _string_to_dts C function, but I am not familiar enough with that code to be able to directly pinpoint the reason.

So it someone want to take a deeper look at this, very welcome!

@jorisvandenbossche jorisvandenbossche added this to the 0.20.0 milestone Nov 7, 2016

@stonebig

This comment has been minimized.

Show comment
Hide comment
@stonebig

stonebig Nov 7, 2016

Contributor

well, as it was during python-3.6.0b3 testing, I reported to python-3.6 team. A false alarm is better than no alarm http://bugs.python.org/issue28636

Contributor

stonebig commented Nov 7, 2016

well, as it was during python-3.6.0b3 testing, I reported to python-3.6 team. A false alarm is better than no alarm http://bugs.python.org/issue28636

@vstinner

This comment has been minimized.

Show comment
Hide comment
@vstinner

vstinner Nov 7, 2016

while with python < 3.6, you get a ValueError:

Python 3.6 probably implements new checks on exceptions.

SystemError: <class 'str'> returned a result with an error set

I suggest to modify pandas to check that you "leak" an exception when calling Python code. Try to add assert(!PyErr_Occurred()); near "pandas/tslib.pyx in pandas.tslib.convert_str_to_tsobject (pandas/tslib.c:26851)()".

vstinner commented Nov 7, 2016

while with python < 3.6, you get a ValueError:

Python 3.6 probably implements new checks on exceptions.

SystemError: <class 'str'> returned a result with an error set

I suggest to modify pandas to check that you "leak" an exception when calling Python code. Try to add assert(!PyErr_Occurred()); near "pandas/tslib.pyx in pandas.tslib.convert_str_to_tsobject (pandas/tslib.c:26851)()".

@jreback jreback modified the milestones: 0.19.2, 0.20.0 Nov 17, 2016

jreback added a commit that referenced this issue Nov 17, 2016

jreback added a commit to jreback/pandas that referenced this issue Nov 17, 2016

@jreback jreback closed this in b52dda8 Nov 17, 2016

amolkahat added a commit to amolkahat/pandas that referenced this issue Nov 26, 2016

ERR: fix exception propogation for datetime parsing functions, noted …
…in python 3.6

closes #14561

Author: Jeff Reback <jeff@reback.net>

Closes #14678 from jreback/py3.6_fix and squashes the following commits:

c8eed83 [Jeff Reback] ERR: fix exception propogation for datetime parsing functions, noted in python 3.6

jorisvandenbossche added a commit to jorisvandenbossche/pandas that referenced this issue Dec 14, 2016

[Backport #14678] ERR: fix exception propogation for datetime parsing…
… functions, noted in python 3.6

closes #14561

Author: Jeff Reback <jeff@reback.net>

Closes #14678 from jreback/py3.6_fix and squashes the following commits:

c8eed83 [Jeff Reback] ERR: fix exception propogation for datetime parsing functions, noted in python 3.6

(cherry picked from commit b52dda8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment