Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

npyio.py --> recfromcsv seems to have a bug that ignores any dtypes provided #311

Closed
kobejohn opened this Issue · 5 comments

4 participants

@kobejohn

I found this due to someone's problem loading large integers with recfromcsv. After playing with it, I think it is simply ignoring all the dtype information provided.

Line 1859 in npyio.py, part of recfromcsv() tries to either bring in the dtypes passed or set None. However, it actually brings in a kwarg that I couldn't find anywhere else called 'update' which looks suspiciously like a copied mistype from earlier in the line. If I change 'update' --> 'dtype', then the types get picked up as expected. More explicitly, in recfromcsv, change:

kwargs.update(dtype=kwargs.get('update', None),

to this:

kwargs.update(dtype=kwargs.get('dtype', None),

I found this in the github repo (although I may have been looking at the wrong branch) as well as the 1.6.2 release that I am using.

(apologies for double posting this. I posted in trac first and then heard that maybe that won't be looked at. I haven't found a way to delete or close that ticket.)

@jarondl jarondl referenced this issue from a commit in jarondl/numpy
Yaron de Leeuw MAINT (API?): organize npyio.recfromcsv defaults
Organizes the default kwargs in recfromcsv. Changes two undocumented
kwargs behaviors:

*  previously, if a user set `names=None`, it was ignored and replaced
   with `names=True`
*  the `dtype` kwarg was ignored. If `update` was given, it was used as
   `dtype`, and if not, None was used. We can retain the `update` behavior
 by using `kwargs.setdefault("dtype",kwargs.get('update', None))`.
  This Closes #311 .
fffb8df
@hpaulj

I came across this issue via a Stackoverflow question:
http://stackoverflow.com/questions/19944408/getting-indices-in-numpy/20172164#20172164

The questioner was loading a file with text fields in Python 3, and having problems testing fields. By default recfromcsv giving these a bytes dtype. Trying to load the file with unicode dtype did not work.

@kobejohn
@hpaulj

kwargs.update(dtype=kwargs.get('dtype', None), solves the problem, as does kwargs.setdefault("dtype", None) (jarondl's patch).

@jarondl

I've added a regression test to #3830 for this bug.

@charris
Owner

@juliantaylor This might fit into your bytes/str work.

@jarondl jarondl referenced this issue from a commit in jarondl/numpy
Yaron de Leeuw MAINT (API?): organize npyio.recfromcsv defaults
Organizes the default kwargs in recfromcsv. Changes two undocumented
kwargs behaviors:

*  previously, if a user set `names=None`, it was ignored and replaced
   with `names=True`
*  the `dtype` kwarg was ignored. If `update` was given, it was used as
   `dtype`, and if not, None was used. We can retain the `update` behavior
 by using `kwargs.setdefault("dtype",kwargs.get('update', None))`.
  This Closes #311 .
803a75f
@charris charris closed this in #3830
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.