Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: dataframe.eval() throws error when using @function if numexpr package is installed #22649

Open
bogdanCsn opened this issue Sep 9, 2018 · 3 comments
Labels
Bug expressions pd.eval, query

Comments

@bogdanCsn
Copy link

bogdanCsn commented Sep 9, 2018

Full details and workaround are available in this SO question

Code Sample, a copy-pastable example if possible

import numpy as np
import pandas as pd
from numpy import around

df = pd.DataFrame({'x':np.array([1.12,2.76])})

# this throws TypeError: 'Series' objects are mutable, thus they cannot be hashed
df['y'] = df.eval('@around(x,1)')

Problem description

The evaluated expression should work because around is imported into the local namespace. There are two solutions:

  • remove numexpr from the environment.
  • set the engine for parsing the expression to be python:
df['y'] = df.eval('@around(x,1)', engine = 'python')

Expected Output

          x    y
0  1.133320  1.1
1  2.733336  2.7

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.7.0.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel byteorder: little LC_ALL: None LANG: en LOCALE: None.None

pandas: 0.23.4
pytest: None
pip: 10.0.1
setuptools: 40.2.0
Cython: None
numpy: 1.15.1
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 6.5.0
sphinx: 1.7.8
patsy: None
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: 2.6.8
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@jreback
Copy link
Contributor

jreback commented Sep 10, 2018

this has nothing to with an import from the local name space or not
eval doesn’t support in line function evaluation

this is a duplicate issue as well

@bogdanCsn
Copy link
Author

@jreback Eval does support in line function evaluation.
I edited my first post to show that there actually is a better solution - specify the parsing engine to be python, no need to remove numexpr from the environment. Running df['y'] = df.eval('@around(x,1)', engine = 'python') produces the expected output.

I'm not sure this is a bug anymore, the workaround to the original issue is trivial.

@bogdanCsn bogdanCsn reopened this Sep 10, 2018
@gfyoung gfyoung added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff DataFrame DataFrame data structure labels Sep 11, 2018
@gfyoung
Copy link
Member

gfyoung commented Sep 11, 2018

this is a duplicate issue as well

@jreback : I'm not entirely sure which issue this duplicates, so labeling for now. Given that the code works with engine='python', is that just coincidental?

@jbrockmendel jbrockmendel added the expressions pd.eval, query label Oct 22, 2019
@mroeschke mroeschke added Bug and removed Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff DataFrame DataFrame data structure labels Apr 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug expressions pd.eval, query
Projects
None yet
Development

No branches or pull requests

5 participants