Skip to content
This repository

npyio.py --> recfromcsv seems to have a bug that ignores any dtypes provided #311

Closed
kobejohn opened this Issue June 16, 2012 · 5 comments

4 participants

John Nieri Paul Jacobson jarondl Charles Harris
John Nieri

I found this due to someone's problem loading large integers with recfromcsv. After playing with it, I think it is simply ignoring all the dtype information provided.

Line 1859 in npyio.py, part of recfromcsv() tries to either bring in the dtypes passed or set None. However, it actually brings in a kwarg that I couldn't find anywhere else called 'update' which looks suspiciously like a copied mistype from earlier in the line. If I change 'update' --> 'dtype', then the types get picked up as expected. More explicitly, in recfromcsv, change:

kwargs.update(dtype=kwargs.get('update', None),

to this:

kwargs.update(dtype=kwargs.get('dtype', None),

I found this in the github repo (although I may have been looking at the wrong branch) as well as the 1.6.2 release that I am using.

(apologies for double posting this. I posted in trac first and then heard that maybe that won't be looked at. I haven't found a way to delete or close that ticket.)

jarondl jarondl referenced this issue from a commit in jarondl/numpy September 29, 2013
MAINT (API?): organize npyio.recfromcsv defaults
Organizes the default kwargs in recfromcsv. Changes two undocumented
kwargs behaviors:

*  previously, if a user set `names=None`, it was ignored and replaced
   with `names=True`
*  the `dtype` kwarg was ignored. If `update` was given, it was used as
   `dtype`, and if not, None was used. We can retain the `update` behavior
 by using `kwargs.setdefault("dtype",kwargs.get('update', None))`.
  This Closes #311 .
fffb8df
Paul Jacobson

I came across this issue via a Stackoverflow question:
http://stackoverflow.com/questions/19944408/getting-indices-in-numpy/20172164#20172164

The questioner was loading a file with text fields in Python 3, and having problems testing fields. By default recfromcsv giving these a bytes dtype. Trying to load the file with unicode dtype did not work.

John Nieri
Paul Jacobson

kwargs.update(dtype=kwargs.get('dtype', None), solves the problem, as does kwargs.setdefault("dtype", None) (jarondl's patch).

jarondl

I've added a regression test to #3830 for this bug.

Charles Harris
Owner

@juliantaylor This might fit into your bytes/str work.

jarondl jarondl referenced this issue from a commit in jarondl/numpy September 29, 2013
MAINT (API?): organize npyio.recfromcsv defaults
Organizes the default kwargs in recfromcsv. Changes two undocumented
kwargs behaviors:

*  previously, if a user set `names=None`, it was ignored and replaced
   with `names=True`
*  the `dtype` kwarg was ignored. If `update` was given, it was used as
   `dtype`, and if not, None was used. We can retain the `update` behavior
 by using `kwargs.setdefault("dtype",kwargs.get('update', None))`.
  This Closes #311 .
803a75f
Charles Harris charris closed this in #3830 March 28, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.