Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grab 'values' attribute from Series in iterate_jit #1470

Merged
merged 1 commit into from
Jul 12, 2017

Conversation

talumbau
Copy link
Member

 - Resolves PSLmodels#1468
 - if attribute passed to 'high level' function is a Series,
   grab the underlying 'values' attribute, which is enough like an
   ndarray to satisfy Numba (so that the jit operation can succeed)
@talumbau
Copy link
Member Author

I don't know why the error described in #1468 manifests now with numpy 1.13. In the release notes, there is this note:

Tuple object dtypes
Support has been removed for certain obscure dtypes that were unintentionally allowed, of the form (old_dtype, new_dtype), where either of the dtypes is or contains the object dtype. As an exception, dtypes of the form (object, [('name', object)]) are still supported due to evidence of existing use.

But I don't know if that is applicable to our case (taking a Pandas Series object and using its 'ndarray-ness' to pass it as an argument to a Numba jitted function).

In the pandas documentation, there is a note about using Pandas data structures with Numba:

https://pandas.pydata.org/pandas-docs/stable/enhancingperf.html

In particular, there is this "scary" blurb near the bottom:

Note As of numba version 0.20, pandas objects cannot be passed directly to numba-compiled functions. Instead, one must pass the numpy array underlying the pandas object to the numba-compiled function as demonstrated below.

But, of course, this project has been doing essentially that for some time. Not sure how we got around it for so long!

This fix may or may not be the best solution, but I think @PeterDSteinberg and others familiar with the PyData stack can see the basic idea. If there are CI issues, it might be some time before I can get back to fixing them, so if others want to use this PR as inspiration for a better fix, that might be the most expedient way to move forward.

@martinholmer
Copy link
Collaborator

@talumbau,
Thanks for your quick explanation and fix of the decorator errors generated when using numpy 1.13. The error report is in #1468 and your ideas for decorator code changes are in #1470.

I'll be happy to test #1470 and fix the PEP8 format warning (so that the GitHub check-in tests are all passed). @PeterDSteinberg is very busy now implementing the new OSPC release process, so I'm happy to handle this from here using your analysis in #1470.

Thanks again!

@MattHJensen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants