Improve memory usage of read_csv, Python objects take up too much

[wesm]

No description provided.

[sanand0]

Wes, was this closed because it's fixed? Does read_csv now work with larger datasets?

[wesm]

It will in 0.10, re: http://wesmckinney.com/blog/?p=543

[fivejjs]

Seem not.
When I do df = pd.read_csv('test.gz', compression='gzip'), it some times fails.

[TomAugspurger]

Hi @fivejjs, this is an old issue.

If you have a specific example that we can reproduce, could you create a new issue with the steps the cause the failure?

[cpcloud]

Also @fivejjs make sure that your issue is related to memory usage. There are all sorts of reasons that your example above could fail. If it's failing at all, please create a new issue.

[snetts]

I am also having the same problem. I am trying to read a CSV file which is approximately 1gb in size and it fails all the time

[kvallinatest]

Is it fixed ?

[jorisvandenbossche]

There is nothing to be 'fixed' here, as this issue is not about a very specific problem. If you have a reproducible use case where you run into troubles with read_csv, please open a new issue.