Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Literal nan string stored in dataframe using json_normalize for null value in string metadata column #25468

Closed
joetl opened this issue Feb 28, 2019 · 3 comments · Fixed by #25619
Labels
IO JSON read_json, to_json, json_normalize Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Milestone

Comments

@joetl
Copy link

joetl commented Feb 28, 2019

Code Sample, a copy-pastable example if possible

# Your code here
import pandas as pd
data = [{"name": "test","address": [{"add": "add1"}]},{"address": [{"add": "add2"}]}]
df = pd.io.json.json_normalize(data,"address","name",errors="ignore")
df

Problem description

In Json_normalize if the metadata column is nullable with option errors set as ignore the null values in dataframe are literal nan string and not numpy.nan

Expected Output

Expecting to see numpy.nan for null metadata string column values instead of nan string

Pandas version 0.24.1

@WillAyd
Copy link
Member

WillAyd commented Feb 28, 2019

Makes sense. Investigation and PRs are always welcome

@WillAyd WillAyd added the IO JSON read_json, to_json, json_normalize label Feb 28, 2019
@WillAyd WillAyd added this to the Contributions Welcome milestone Feb 28, 2019
@antoineviscardi
Copy link
Contributor

Hi there,

I can take care of this. Would we want this to be the only behavior or optional by introducing a new named parameter?

@WillAyd
Copy link
Member

WillAyd commented Mar 9, 2019

@antoineviscardi this should be the only behavior

@gfyoung gfyoung added the Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate label Mar 9, 2019
@gfyoung gfyoung modified the milestone: Contributions Welcome Mar 9, 2019
@jreback jreback modified the milestones: Contributions Welcome, 0.25.0 Mar 10, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO JSON read_json, to_json, json_normalize Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants