Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas concat working in 0.12 but not 0.13 #5754

Closed
MichaelWS opened this issue Dec 20, 2013 · 6 comments · Fixed by #5757
Closed

pandas concat working in 0.12 but not 0.13 #5754

MichaelWS opened this issue Dec 20, 2013 · 6 comments · Fixed by #5757
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@MichaelWS
Copy link
Contributor

I have a long list of dataframes that concat fine in 0.12 but not in 0.13. Has anyone seen this error?
I am concatenating using the following

pd.concat(all_gics, ignore_index=True) 

My traceback:

/usr/anaconda/lib/python2.7/site-packages/pandas/tools/merge.pyc in concat(objs, axis, join, join_axes, ignore_index, keys, levels, names, verify_integrity)
925 keys=keys, levels=levels, names=names,
926 verify_integrity=verify_integrity)
--> 927 return op.get_result()
928
929

/usr/anaconda/lib/python2.7/site-packages/pandas/tools/merge.pyc in get_result(self)
1016 return tmpdf
1017 else:
-> 1018 new_data = self._get_concatenated_data()
1019 new_data = self._post_merge(new_data)
1020 return self.objs[0]._from_axes(new_data, self.new_axes)

/usr/anaconda/lib/python2.7/site-packages/pandas/tools/merge.pyc in _get_concatenated_data(self)
1075 new_data = {}
1076 for item in self.new_axes[0]:
-> 1077 new_data[item] = self._concat_single_item(rdata, item)
1078
1079 return new_data

/usr/anaconda/lib/python2.7/site-packages/pandas/tools/merge.pyc in _concat_single_item(self, objs, item)
1184 elif isnull(v).all():
1185
-> 1186 alls.remove('other')
1187 alls.remove('object')
1188

KeyError: 'other'

@jreback
Copy link
Contributor

jreback commented Dec 20, 2013

pls put up a reproducible example

@MichaelWS MichaelWS reopened this Dec 20, 2013
@MichaelWS
Copy link
Contributor Author

I will try to create an example that mimics it. (Its a private data source or else I would immediately post it immediately.)

@jreback
Copy link
Contributor

jreback commented Dec 20, 2013

@MichaelWS I have reproduced the problem; was really tested code, try with PR #5757

I believe you were appending row-wise a datetime64[ns] dtype and something else (with a non-empty type, say a nan / string)

@MichaelWS
Copy link
Contributor Author

This now works and you are right. Thanks

@jreback
Copy link
Contributor

jreback commented Dec 20, 2013

gr8

in the future if cannot submit a link to your actual data just make a copy of what reproduces it and replace with transformed/modified data

and as small a set as u can

it helps in finding the errors

thanks for the reports

@MichaelWS
Copy link
Contributor Author

Thanks, I will post a sample going forward.

I would have mocked up a sample but I wasn't sure what the issue was.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants