Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas.Series.tolist should convert Series contents from numpy datatypes to native Python datatypes #10904

Closed
cglwn opened this issue Aug 25, 2015 · 1 comment
Labels
Compat pandas objects compatability with Numpy or Python functions Dtype Conversions Unexpected or buggy dtype conversions Enhancement
Milestone

Comments

@cglwn
Copy link

cglwn commented Aug 25, 2015

Noticed this in Pandas v0.16.1 when JSON serializing a large nested dictionary containing mixes of arrays, numpy arrays, and pandas series. Calling tolist() on numpy arrays allows the standard library JSON serializer to serialize them, but this is not the case for pandas.

numpy.array.tolist converts all of the array's contents to native Python datatypes. For compatibility with numpy, pandas.Series.tolist should also convert its contents to native Python datatypes. Currently, the list returned by pandas contains elements which have types like numpy.int64 or numpy.float64.

For example:

>>> from pandas
>>> import numpy
>>> np_array = numpy.array([1, 2, 3])
>>> pd_series = pandas.Series([1,2,3])
>>> np_list = np_array.tolist()
>>> pd_list = pd_series.tolist()
>>> type(np_list[0])
<class 'int'>
>>> type(pd_list[0])
<class 'numpy.int64'>
@jreback
Copy link
Contributor

jreback commented Aug 26, 2015

this is a bit tricker than it looks. pandas correctly handles boxing of dtypes (e.g. to Timestamp/categorical and such). So this is really just list(self). I suppose you could box integer/floats. Though IMHO this is a pretty useless method and really just for numpy compat.

If you'd like to do a pull-request would prob accept.

you should simply use .to_json()

@jreback jreback added Enhancement Prio-low Dtype Conversions Unexpected or buggy dtype conversions Compat pandas objects compatability with Numpy or Python functions labels Aug 26, 2015
@jreback jreback modified the milestones: 1.0, Next Major Release Aug 26, 2015
@jreback jreback modified the milestones: 0.18.2, Next Major Release May 6, 2016
@jreback jreback closed this as completed in b2009e6 May 7, 2016
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Dtype Conversions Unexpected or buggy dtype conversions Enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants