You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have fixed width text files with headers, the size of which (the amount of lines to skip) varies greatly from file to file. I could first read a file and figure out the amount of lines to skip before reloading it with the skip value set, but it feels clumsy and it would be nice to be able to do that in one go. Headers/Comments being often identifiable by the first few characters (in my case, the line starts by the letter H).
It would also be nice to have the possibility to save those skipped lines (the whole header) in a variable or a file at the same time as one may need to parse it further to collect some information from it.
I was about to say that the new-ish argument comment should work for the header skipping (as long as data lines don't start with H). See #68, now closed/fixed. But now I see that comment is not (yet?) available for read_fwf().
Something like comment could definitely work, yes.
It may be worth nothing though that in the case of a header, it is not necessary to test every single line as it is clear that after we've encountered the first row of data, there is no more header.
Two suggestions for interested parties (not just Hadley!)
Assign comments to the returned object using 'comment()<-'.
Design an API for a more general 2-part approach. Maybe 'meta=list(prefix="#", parser=as.character, simplify=FALSE)' would accomplish the above? Support for emerging CSV metadata conventions could be implemented as extensions.
Activity
jennybc commentedon Dec 12, 2015
I was about to say that the new-ish argument
commentshould work for the header skipping (as long as data lines don't start with H). See #68, now closed/fixed. But now I see thatcommentis not (yet?) available forread_fwf().yportier commentedon Dec 14, 2015
Something like
commentcould definitely work, yes.It may be worth nothing though that in the case of a header, it is not necessary to test every single line as it is clear that after we've encountered the first row of data, there is no more header.
dholstius commentedon May 19, 2016
Two suggestions for interested parties (not just Hadley!)
[-]Feature Request: conditional skip[/-][+]Add comment argument to `read_fwf()`[/+][-]Add comment argument to `read_fwf()`[/-][+]Implement comment skipping for read_fwf()[/+]hadley commentedon Jun 2, 2016
@holstius that's unfortunately rather difficult to do without losing the performance benefits of the way that readr is structured.