BUG: Standardize malformed row handling in Python engine #15913

Merged
merged 1 commit into from Apr 6, 2017

Conversation

Projects
None yet
3 participants
Member

gfyoung commented Apr 6, 2017

Wrap next(source) when source is a file-buffer object to allow for more uniform error message displaying in the Python engine. Closes #15910.

pandas/io/parsers.py
+ elif 'newline inside string' in msg:
+ msg = ('EOF inside string starting with '
+ 'line ' + str(kwargs['row_num']))
+ raise Exception(msg)
@jreback

jreback Apr 6, 2017

Contributor

this is a plain Exception? shouldn't it be a ValueError?

@gfyoung

gfyoung Apr 6, 2017 edited

Member

Copied from where it was originally, which according to the blame was refactored by you! 😄

In any case, I agree: we should just remove that line and raise csv.Error now

@jreback

jreback Apr 6, 2017

Contributor

hmm don't remember that :)

@jreback

jreback Apr 6, 2017

Contributor

yeah csv.Error sounds fine

make a note of this in the whatsnew though

@gfyoung

gfyoung Apr 6, 2017

Member

Yep, done.

codecov bot commented Apr 6, 2017 edited

Codecov Report

Merging #15913 into master will increase coverage by <.01%.
The diff coverage is 95.65%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #15913      +/-   ##
==========================================
+ Coverage   90.95%   90.96%   +<.01%     
==========================================
  Files         145      145              
  Lines       49534    49531       -3     
==========================================
- Hits        45056    45055       -1     
+ Misses       4478     4476       -2
Flag Coverage Δ
#multiple 88.72% <95.65%> (ø) ⬆️
#single 40.63% <4.34%> (-0.03%) ⬇️
Impacted Files Coverage Δ
pandas/io/parsers.py 95.65% <95.65%> (+0.12%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba30e3a...884387e. Read the comment docs.

@gfyoung gfyoung BUG: Standardize malformed row handling in Python engine
Closes gh-15910.
884387e
Member

gfyoung commented Apr 6, 2017

@jreback : Everything is green and ready to go.

@jreback jreback added IO CSV Bug labels Apr 6, 2017

jreback added this to the 0.20.0 milestone Apr 6, 2017

@jreback jreback merged commit a0b089e into pandas-dev:master Apr 6, 2017

5 checks passed

ci/circleci Your tests passed on CircleCI!
Details
codecov/patch 95.65% of diff hit (target 50%)
Details
codecov/project 90.96% (target 82%)
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Contributor

jreback commented Apr 6, 2017

thanks @gfyoung

gfyoung deleted the gfyoung:invalid-quote-skipfooter branch Apr 6, 2017

@linebp linebp added a commit to linebp/pandas that referenced this pull request Apr 17, 2017

@gfyoung @linebp gfyoung + linebp BUG: Standardize malformed row handling in Python engine (#15913)
Closes gh-15910.
43eb7d6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment