New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include filename in warning message when parsing of values fails #581

Closed
estroger34 opened this Issue Jan 20, 2017 · 6 comments

Comments

3 participants
@estroger34

estroger34 commented Jan 20, 2017

I read a large number of files using lapply and read_csv with fixed coltypes:

datalist <- lapply(filelist, function (x) read_csv(x,col_types='ccdddic')

Occasionally there is a non-numeric entry in a column that should parse as double and I get a warning message like this:

Warning: 1 parsing failure.
row    col expected actual
  1 Wert01 a double      b 

However, from this warning I do not know in which of the dozens, sometimes hundreds of files the parsing failed.

Would it be possible to include the filename for the failed parsing in the warning message?

@hadley

This comment has been minimized.

Member

hadley commented Jan 20, 2017

I agree that this is good practice

@hadley

This comment has been minimized.

Member

hadley commented Jan 20, 2017

Ideally we should also include the filename if there are any errors

@jimhester jimhester added this to TODO in jimhester Feb 2, 2017

@jimhester

This comment has been minimized.

Member

jimhester commented Feb 7, 2017

I started working on this, it is going to require quite extensive changes to implement as none of the collectors or parsers store the filename information. We need to think about what internal objects should store the filename information and which should not. In particular I am pretty sure we need to avoid including the filename in the Token object, as that would likely slow parsing considerably.

@jimhester jimhester moved this from TODO to BACKLOG in jimhester Feb 7, 2017

@hadley

This comment has been minimized.

Member

hadley commented Feb 7, 2017

What if we did this at the R level by wrapping the condition raised in C++?

@jimhester

This comment has been minimized.

Member

jimhester commented Feb 7, 2017

That would work, I can give it a try.

jimhester added a commit that referenced this issue Feb 7, 2017

jimhester added a commit that referenced this issue Feb 7, 2017

@jimhester

This comment has been minimized.

Member

jimhester commented Feb 7, 2017

Turns out it was even easier than using exceptions, we can just add the filename to the problems attribute directly, should have thought of that first!

@jimhester jimhester moved this from BACKLOG to DONE in jimhester Feb 7, 2017

jimhester added a commit that referenced this issue Feb 8, 2017

jimhester added a commit that referenced this issue Feb 8, 2017

@lock lock bot locked and limited conversation to collaborators Sep 24, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.