New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to specify header values and ignore headers in the csv file #27

Closed
wsmoak opened this Issue Feb 14, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@wsmoak

wsmoak commented Feb 14, 2016

Thanks for the CSV library!

In order to use the Map parsed from each row with Ecto, I need to define the headers as atoms that exactly match my model:

File.stream!("/Users/wsmoak/Downloads/transactions.csv")
  |> CSV.decode(headers: [:date, :description, :original_description, :amount, :transaction_type, :category, :account_name, :labels, :notes])

When I do this, the first line of the file that contains the headers is picked up as a row of values, which I don't want.

Is there a way to skip that first row the way headers: false would, and define the values for the headers for the map?

Would having two different attributes work better? Maybe :headers could be true/false only, to say whether the csv file contains headers, and then a separate attribute could be used to optionally specify the header values to use?

(My workaround is to define a function that matches on one of the things I know will be in the Map for that first unwanted record, and ignore it: def store_it(%{:date => "Date"}) do ... end )

@beatrichartz

This comment has been minimized.

Show comment
Hide comment
@beatrichartz

beatrichartz Feb 14, 2016

Owner

Interesting usecase, thanks for raising this!

I think in this case the solution could be done outside the CSV module by piping in Stream.drop(1) before feeding the stream into decode:

File.stream!("/Users/wsmoak/Downloads/transactions.csv")
  |> Stream.drop(1)
  |> CSV.decode(headers: [:date, :description, :original_description, :amount, :transaction_type, :category, :account_name, :labels, :notes])

The library does not filter or skip rows in any case - headers: false lets the library read the first row as a data row, headers: true reads it as a headers row. I quite like having any skipping or filtering done before feeding the stream into the library since it makes the intention clear and keeps these decisions out of the library.

Hope this works for you, feel free to reopen if you have more questions / this is turning into a bug!

Owner

beatrichartz commented Feb 14, 2016

Interesting usecase, thanks for raising this!

I think in this case the solution could be done outside the CSV module by piping in Stream.drop(1) before feeding the stream into decode:

File.stream!("/Users/wsmoak/Downloads/transactions.csv")
  |> Stream.drop(1)
  |> CSV.decode(headers: [:date, :description, :original_description, :amount, :transaction_type, :category, :account_name, :labels, :notes])

The library does not filter or skip rows in any case - headers: false lets the library read the first row as a data row, headers: true reads it as a headers row. I quite like having any skipping or filtering done before feeding the stream into the library since it makes the intention clear and keeps these decisions out of the library.

Hope this works for you, feel free to reopen if you have more questions / this is turning into a bug!

@wsmoak

This comment has been minimized.

Show comment
Hide comment
@wsmoak

wsmoak Feb 14, 2016

Thanks, that works!

wsmoak commented Feb 14, 2016

Thanks, that works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment