Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pd.concat merges object-datetime and np.datetime columns incorrectly. #2624

Closed
dalejung opened this issue Jan 1, 2013 · 2 comments
Closed
Assignees
Labels
Milestone

Comments

@dalejung
Copy link
Contributor

dalejung commented Jan 1, 2013

rows = []
rows.append([pd.datetime(2010, 1, 1), 1])
rows.append([pd.datetime(2010, 1, 2), 'hi']) # test col upconverts to obj

df2_obj = pd.DataFrame.from_records(rows, columns=['date', 'test'])

ind = pd.date_range(start="2000/1/1", freq="D", periods=10)
df1 = pd.DataFrame({'date': ind, 'test':range(10)})

cdf = pd.concat([df1, df2_obj])
cdf

The rows where date was a np.datetime are converted to ints, while the object dates are left as datetimes.

I'm not sure if this is a bug. I would expect the np.datetime column to upconvert into Timestamp objects?

I ran into this from a test failure. Thing is, it's a fairly old test and it used to pass. I checked it against 0.9.1 and 0.10.0 which failed as well. So I'm not sure if I was on a previous commit that worked differently. it's weird.

Either way, just wanted to get a check on what the expected behavior is.

EDIT: http://nbviewer.ipython.org/4428511/

@dalejung
Copy link
Contributor Author

dalejung commented Jan 3, 2013

Just a note. This isn't an exact replica of my unit test. In my code, df1.date is created with a list of Timestamps. Which is why it was working before. Prior to e06334a
all of the date columns would've remained dtype(object).

@wesm
Copy link
Member

wesm commented Jan 20, 2013

fixed, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants