Skip to content

Implement comment skipping for read_fwf() #334

Closed

Description

@yportier

I have fixed width text files with headers, the size of which (the amount of lines to skip) varies greatly from file to file. I could first read a file and figure out the amount of lines to skip before reloading it with the skip value set, but it feels clumsy and it would be nice to be able to do that in one go. Headers/Comments being often identifiable by the first few characters (in my case, the line starts by the letter H).
It would also be nice to have the possibility to save those skipped lines (the whole header) in a variable or a file at the same time as one may need to parse it further to collect some information from it.

Activity

  1. jennybc commented on Dec 12, 2015

    @jennybc
    Member

    I was about to say that the new-ish argument comment should work for the header skipping (as long as data lines don't start with H). See #68, now closed/fixed. But now I see that comment is not (yet?) available for read_fwf().

  2. yportier commented on Dec 14, 2015

    @yportier
    Author

    Something like comment could definitely work, yes.
    It may be worth nothing though that in the case of a header, it is not necessary to test every single line as it is clear that after we've encountered the first row of data, there is no more header.

  3. dholstius commented on May 19, 2016

    @dholstius

    Two suggestions for interested parties (not just Hadley!)

    1. Assign comments to the returned object using 'comment()<-'.
    2. Design an API for a more general 2-part approach. Maybe 'meta=list(prefix="#", parser=as.character, simplify=FALSE)' would accomplish the above? Support for emerging CSV metadata conventions could be implemented as extensions.
  4. changed the title [-]Feature Request: conditional skip[/-] [+]Add comment argument to `read_fwf()`[/+] on Jun 2, 2016
  5. changed the title [-]Add comment argument to `read_fwf()`[/-] [+]Implement comment skipping for read_fwf()[/+] on Jun 2, 2016
  6. hadley commented on Jun 2, 2016

    @hadley
    Member

    @holstius that's unfortunately rather difficult to do without losing the performance benefits of the way that readr is structured.

  7. modified the milestone: 0.3.0 on Jul 13, 2016
  8. self-assigned this
    on Jul 14, 2016
  9. locked and limited conversation to collaborators on Sep 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

featurea feature request or enhancement

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions