New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

json_normalize throws `TypeError` with array of values and `record_prefix` #21536

Closed
vuminhle opened this Issue Jun 19, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@vuminhle
Contributor

vuminhle commented Jun 19, 2018

Code Sample, a copy-pastable example if possible

from pandas.io.json import json_normalize

df = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
print(df)

Problem description

The above code throws a TypeError:

Traceback (most recent call last):
  File "c:\Users\levu\Desktop\tmp\json_normalize\main.py", line 3, in <module>
    df = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 262, in json_normalize
    result.rename(columns=lambda x: record_prefix + x, inplace=True)
  File "C:\Python36\lib\site-packages\pandas\util\_decorators.py", line 187, in wrapper
    return func(*args, **kwargs)
  File "C:\Python36\lib\site-packages\pandas\core\frame.py", line 3781, in rename
    return super(DataFrame, self).rename(**kwargs)
  File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 973, in rename
    level=level)
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 3340, in rename_axis
    obj.set_axis(axis, _transform_index(self.axes[axis], mapper, level))
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 5298, in _transform_index
    items = [func(x) for x in index]
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 5298, in <listcomp>
    items = [func(x) for x in index]
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 262, in <lambda>
    result.rename(columns=lambda x: record_prefix + x, inplace=True)
TypeError: must be str, not int

I think line 262 in normalize.py should be:

result.rename(columns=lambda x: "{p}{c}".format(p=record_prefix,c=x), inplace=True)

because x can be integer.

Expected Output

Prefix.0
0 1
1 2

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.1
pytest: 3.6.1
pip: 10.0.1
setuptools: 28.8.0
Cython: None
numpy: 1.14.2
scipy: None
pyarrow: None
xarray: None
IPython: 6.3.1
sphinx: None
patsy: None
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@gfyoung

This comment has been minimized.

Member

gfyoung commented Jun 19, 2018

That indeed looks suspicious! PR to patch is welcome!

cc @WillAyd

@gfyoung gfyoung added this to the 0.23.2 milestone Jun 19, 2018

@gfyoung

This comment has been minimized.

Member

gfyoung commented Jun 19, 2018

@vuminhle : Marking this for 0.23.2, as @vuminhle has already identified a potential fix, which we can easily check and patch if this actually works if no one picks this up.

vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018

vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018

vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018

vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018

jreback added a commit that referenced this issue Jun 22, 2018

jorisvandenbossche added a commit that referenced this issue Jun 29, 2018

jorisvandenbossche added a commit that referenced this issue Jul 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment