TextFileReader.get_chunk returns full file DataFram despite of chunksize specified in read_csv

[vshkolyar]

TextFileReader.get_chunk returns full file DataFram despite of chunksize specified in read_csv.

[jreback]

can you show exactly what you are doing? you need to specify the number of rows that you want in get_chunk, otherwise it returns the full frame. normally you would just iterator over the returned object anyhow.

http://pandas.pydata.org/pandas-docs/dev/io.html#iterating-through-files-chunk-by-chunk

[vshkolyar]

read_csv method called with some specific chunk size.
When I call afterwards get_chunk method there is two cases:

1. chunksize parameter passed to the method - I expect to receive chunk with size specified as argument in the method that overrides the instance chunk size the TextFileReader initialized with (passed to read_csv as parameter), and this OK.
2. But, in case no such parameter passed to the get_chunk, I would expect to receive DataFrame with chunk size specified in read_csv, that TextFileReader instance initialized with and stored as instance variable (property). Again, that because get_chunk is type's instance method (not static type method, not some global function), and this instance of this type holds the chunksize member inside.

P.S. So I can call my_text_file_reader.get_chunk(my_text_file_reader.chunksize), but this is ugly and not object oriented.

[jreback]

@wes this is expected behavior yes?
IOW get_chunk allows u to get variable sizes chunks by calling with a parameter, or everything if no parameter? or should default to passed chunk size ?

[wesm]

Calling it a bug. Fixed, thanks