Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: 'Unnamed' != unnamed column in CSV #23687

Merged
merged 1 commit into from Nov 16, 2018

Conversation

gfyoung
Copy link
Member

@gfyoung gfyoung commented Nov 14, 2018

False criterion was causing errors when specified headers appeared to capture
a seemingly unnamed row, just because they had the string "Unnamed" in it.

Setup:

from pandas import read_csv
from pandas.compat import StringIO

data = "Unnamed,NotUnnamed\n0,1\n2,3\n4,5"
read_csv(StringIO(data), header=[0, 1])

Previously, this would error:

...
ValueError : Passed header=[0,1] are too many rows for this multi_index of columns

Now, it nicely returns a DataFrame:

  Unnamed NotUnnamed
        0          1
0       2          3
1       4          5

Leverages the patch used in #23484 of self.unnamed_cols.

@pep8speaks
Copy link

Hello @gfyoung! Thanks for submitting the PR.

@gfyoung gfyoung added Bug IO CSV read_csv, to_csv labels Nov 14, 2018
@gfyoung gfyoung added this to the 0.24.0 milestone Nov 14, 2018
@codecov
Copy link

codecov bot commented Nov 14, 2018

Codecov Report

Merging #23687 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #23687      +/-   ##
==========================================
- Coverage   92.25%   92.25%   -0.01%     
==========================================
  Files         161      161              
  Lines       51383    51381       -2     
==========================================
- Hits        47404    47402       -2     
  Misses       3979     3979
Flag Coverage Δ
#multiple 90.64% <100%> (-0.01%) ⬇️
#single 42.32% <0%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/io/parsers.py 95.54% <100%> (-0.01%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a23f901...ff68dbd. Read the comment docs.

@gfyoung
Copy link
Member Author

gfyoung commented Nov 14, 2018

@jreback : Address the comment, and all is still green. PTAL.

@jreback
Copy link
Contributor

jreback commented Nov 14, 2018

conflict in the whatsnew, can you merge master

@gfyoung
Copy link
Member Author

gfyoung commented Nov 15, 2018

@jreback : Rebased my changes, and all is green. PTAL.

pandas/io/parsers.py Outdated Show resolved Hide resolved
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
@gfyoung
Copy link
Member Author

gfyoung commented Nov 15, 2018

@jreback : Addressed all comments, all is still green. PTAL.

@jreback jreback merged commit 4a6a918 into pandas-dev:master Nov 16, 2018
@jreback
Copy link
Contributor

jreback commented Nov 16, 2018

thanks!

@gfyoung gfyoung deleted the unnamed-unrobust-criterion branch November 16, 2018 21:27
tm9k1 pushed a commit to tm9k1/pandas that referenced this pull request Nov 19, 2018
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
False criterion was causing errors when
specified headers appeared to capture
a seemingly unnamed row, just because
they had the string "Unnamed" in it.
@SamLevinSE
Copy link

I'm getting bunch of these "unnamed ..." where my column is empty. Is this related to the same bug?
ex:

Unnamed: 1_level_0 Baseline Features Unnamed: 3_level_0 Unnamed: 4_level_0 Unnamed: 5_level_0 Unnamed: 6_level_0 Unnamed: 7_level_0

@gfyoung
Copy link
Member Author

gfyoung commented Feb 27, 2020

@SamLevinSE : I don't believe so. However, if you are welcome to open a new issue to provide more details, as it could still be a problem with pandas (can't tell from this).

@SamLevinSE
Copy link

@gfyoung What if I give you some more Information? could you please check these two screenshots and let me know if I need to create a new bug/issue or if I'm doing something wrong?
image

image
Thank you!

@gfyoung
Copy link
Member Author

gfyoung commented Feb 27, 2020

This looks expected as far as I can see.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants