Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError in Cython with tuple subclasses in DataFrame.from_records #611

wesm opened this issue Jan 11, 2012 · 2 comments


Copy link

commented Jan 11, 2012

From user email

Hi there,

Just a quick bug report. I'm using psycopg2 with a NamedTupleCursor, which returns namedtuples for rows. I'm trying to use DataFrame.from_records like so:

df = DataFrame.from_records(rows, rows[0]._fields)

I get the following error:

TypeError                                 Traceback (most recent call last)
/Users/kdevens/source/(etc.) <ipython-input-46-3e705b917abc> in <module>()
----> 1 df = DataFrame.from_records(rows, rows[0]._fields)

/Library/Python/2.6/site-packages/pandas-0.6.1-py2.6-macosx-10.6-universal.egg/pandas/core/frame.pyc in from_records(cls, data, index, exclude, columns, names)
    533             columns, sdict = _rec_to_dict(data)
    534         else:
--> 535             sdict, columns = _list_to_sdict(data, columns)
    537         if exclude is None:

/Library/Python/2.6/site-packages/pandas-0.6.1-py2.6-macosx-10.6-universal.egg/pandas/core/frame.pyc in _list_to_sdict(data, columns)
   3547 def _list_to_sdict(data, columns):
   3548     if isinstance(data[0], tuple):
-> 3549         content = list(lib.to_object_array_tuples(data).T)
   3550     else:
   3551         # list of lists

/Library/Python/2.6/site-packages/pandas-0.6.1-py2.6-macosx-10.6-universal.egg/pandas/ in pandas._tseries.to_object_array_tuples (pandas/src/tseries.c:52160)()

TypeError: Expected tuple, got Record

So it looks like in tseries.c it's doing something like "type(row) == tuple" instead of "isinstance(row, tuple)". For what it's worth, "isinstance(rows[0], tuple)" is true, so the tuples returned by psycopg2 are definitely real namedtuples. Additionally, the following works fine:

In [49]: dictrows = [r._asdict() for r in rows]

In [50]: df = DataFrame.from_dict(dictrows)

So there's no problem with the data, or with DataFrame, just the type check in tseries.c.

Let me know if I can provide any more information! Thanks.

@ghost ghost assigned adamklein Jan 12, 2012


This comment has been minimized.

Copy link

commented Jan 12, 2012

I hope I solved this one. @wesm, will leave to you to close.

wesm added a commit that referenced this issue Jan 12, 2012

This comment has been minimized.

Copy link
Member Author

commented Jan 12, 2012

Tweaked one thing, everything looks good now

@wesm wesm closed this Jan 12, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
2 participants
You can’t perform that action at this time.