Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: read_fwf() does not obey skip_blank_lines #37758

Closed
dnicolodi opened this issue Nov 11, 2020 · 0 comments · Fixed by #37803
Closed

BUG: read_fwf() does not obey skip_blank_lines #37758

dnicolodi opened this issue Nov 11, 2020 · 0 comments · Fixed by #37803
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member
Milestone

Comments

@dnicolodi
Copy link
Contributor

dnicolodi commented Nov 11, 2020

The read_fwf() function does not obey the skip_blank_lines argument:

data = io.StringIO("""\
 1 2 3
 4 5 6
""")

d = pandas.read_fwf(data, header=None, colspec=[(0, 2), (2, 4), (4, 6)])

results in:

   0  1  2
0  1  2  3
1  4  5  6

while

data = io.StringIO("""\
 1 2 3

 4 5 6
""")

d = pandas.read_fwf(data, header=None, colspec=[(0, 2), (2, 4), (4, 6)])

results in

     0    1    2
0  1.0  2.0  3.0
1  NaN  NaN  NaN
2  4.0  5.0  6.0

The skip_blank_lines argument defaults to True, but also explicitly setting it results in the same behavior. The read_fwf() function documentation says that it takes all arguments supported by TextFileReader thus this is unexpected.

I have tested this with Pandas 1.1.3 as distributed by conda, however, the relevant code does not seem to have changed on master.

@dnicolodi dnicolodi added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Nov 11, 2020
@jreback jreback added this to the 1.2 milestone Nov 14, 2020
gfyoung pushed a commit that referenced this issue Nov 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants