Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to handle fractional seconds in date_converters #2209

Closed
wesm opened this issue Nov 9, 2012 · 5 comments

Comments

@wesm
Copy link
Member

commented Nov 9, 2012

cc @michaelaye

I have a 'second' column that contains microseconds like these:
In [52]: df.second.values
Out[52]: 
array([  4.50000000e-02,   4.50000000e-02,   4.50000000e-02, ...,
         5.99920000e+01,   5.99920000e+01,   5.99920000e+01])

and had used this column to produce a DateTime index like this before:

    d = dataframe
    di = pandas.io.date_converters.parse_all_fields(
    d.year, d.month, d.date, d.hour, d.minute, d.second)

but the resulting datetime index lost the microseconds:

MultiIndex
[(1.0, 1.0, <Timestamp: 2009-07-20 00:00:00>)
 (1.0, 2.0, <Timestamp: 2009-07-20 00:00:00>)
 (1.0, 3.0, <Timestamp: 2009-07-20 00:00:00>) ...,
 (9.0, 19.0, <Timestamp: 2009-07-20 00:59:59>)
 (9.0, 20.0, <Timestamp: 2009-07-20 00:59:59>)
 (9.0, 21.0, <Timestamp: 2009-07-20 00:59:59>)]

Is there a way to keep the microseconds?
@michaelaye

This comment has been minimized.

Copy link
Contributor

commented Nov 19, 2012

You said in your tutorial that you want to support up to nanosecond time precision. But as I see, you are using the standard datetime.datetime class for parsing times, where it does not seem possible, as microseconds has to be an int as well. How should I fix it? String split the seconds string in 2 entities by the '.' and provide it to datetime?

michaelaye pushed a commit to michaelaye/pandas that referenced this issue Nov 19, 2012
@michaelaye

This comment has been minimized.

Copy link
Contributor

commented Nov 19, 2012

What about that? Of course string parsing must be too slow, so I tried to avoid it. Tell me if I should do something different.

michaelaye pushed a commit to michaelaye/pandas that referenced this issue Nov 19, 2012
@michaelaye

This comment has been minimized.

Copy link
Contributor

commented Nov 19, 2012

FYI: I confirmed that this does not fail with integer seconds that do not have microseconds data.

@wesm wesm closed this in 7306b65 Nov 30, 2012

@wesm

This comment has been minimized.

Copy link
Member Author

commented Nov 30, 2012

I wrote a test case and made this work. thanks!

@michaelaye

This comment has been minimized.

Copy link
Contributor

commented Dec 1, 2012

Great, thanks. Sorry I did not respond earlier for the test cases. But as I see, you made a better solution anyway...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.