Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/csv: need more functionalities in package #33237

Open
Prithvipal opened this issue Jul 23, 2019 · 6 comments
Open

encoding/csv: need more functionalities in package #33237

Prithvipal opened this issue Jul 23, 2019 · 6 comments

Comments

@Prithvipal
Copy link

@Prithvipal Prithvipal commented Jul 23, 2019

The csv package has very basic functionalities like Read(): reads single record, ReadAll() read all records etc.
Since csv always has list of column names. All records are always corresponding to column names.

Proposal

There should be some functionalities related to Columns(not expecting big as Pandas) so it will be easier to process csv files using built-in package itself.

  1. Get column by column name so the caller can understand which are the column available in cvs
  2. Delete Columns
  3. Filter by Column
  4. List all columns
  5. Add new column
    etc.
@sachi-gkp
Copy link

@sachi-gkp sachi-gkp commented Jul 24, 2019

I agree on this proposal.
Currently, enconding/csv package provides very basic features which is not useful to write production grade code. And developer needs to write too much boilerplate codes in order to use this package..

@nussjustin
Copy link
Contributor

@nussjustin nussjustin commented Jul 24, 2019

The encoding/csv package already provides support for reading and writing of CSV. Everything else can be easily build on top of this, so I don't see why any of this should be in the stdlib.

What speaks against providing this functionality in a third party package?

I've worked with many different CSV files in the last few years, but never needed any of the features you listed except the first one (which can be easily solved with a simple map[string]string and can be abstracted away in just a few lines of code).

Also you say that CSV always has column names. This is wrong. Though relatively rare, there are still many CSV files without column names (often these names are just not in the CSV file but are always the same, so they will be hardcoded in the application).

Now even if there are good reasons for having this in the stdlib the new functionality could still be implemented outside the stdlib first and then later adopted into the stdlib (or one of the golang.org/x repositories). That way it's much easier to iterate on the API and collecting feedback from users (once adopted into the stdlib, we can't break backwards compatibility)

Also please try to give examples (e.g. function signatures, types, ...) instead of just listing ideas for features.

@julieqiu julieqiu changed the title encoding/csv: Need more functionalities in package encoding/csv: need more functionalities in package Jul 24, 2019
@julieqiu
Copy link
Contributor

@julieqiu julieqiu commented Jul 24, 2019

/cc @dsnet

@dsnet
Copy link
Member

@dsnet dsnet commented Aug 5, 2019

The encoding/csv package fundamentally operates on a CSV file in a streaming manner. For this reason it takes in an io.Reader and an io.Writer. However, the functionality that this issue proposes requires:

  1. that the entire CSV file to be loaded into memory (this is in stark contrast to how the encoding/csv package currently operates).
  2. that there is a new type that represents an in-memory representation of the CSV file (e.g., a named [][]interface{} type) that you can perform the filter operations on.

While I can see such filter operations being useful, I don't see why

  1. it needs to be in the standard library. If so, encoding/csv is the wrong place for it since it actually has nothing to do with encoding since it's about filtering and querying the data after serialization.
  2. why it needs to be about CSV. You can imagine such filter/query functionality be easily expand to operate on JSON-like data structures or really any generic Go data structure.

This functionality seems like it should be explored outside the standard library.

@Prithvipal
Copy link
Author

@Prithvipal Prithvipal commented Aug 7, 2019

encoding/csv is the wrong place for it since it actually has nothing to do with encoding

We can introduce this functionality in x package.

You can imagine such filter/query functionality be easily expand to operate on JSON-like data structures

I agree with this. This functionality can be implemented on any data structure like JSON.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
@sachi-gkp @julieqiu @dsnet @nussjustin @Prithvipal and others
You can’t perform that action at this time.