BUG: query with invalid dtypes should fallback to python engine #10486

Closed
DSLituiev opened this Issue Jul 1, 2015 · 1 comment

Comments

Projects
None yet
2 participants

The following call of df.query() produces an error:

...
File "/Applications/anaconda/lib/python3.4/site-packages/numexpr/necompiler.py", line 629, in getType
raise ValueError("unkown type %s" % a.dtype.name)

ValueError: unkown type str128

in pandas 0.15.2

# -*- coding: utf-8 -*-
import re
import pandas as pd

def get_gene_site(tp):
    site = re.sub('T.$', '',tp)
    gene = site.split('%%')[0]
    t = int(re.search(r'T(\d+)$',tp).group(1))
    return gene, site, t

sub_sample_list = ['Actb%%qeqwT0',
 'Actb%%qeqwT1',
 'Actb%%qeqwT2',
 'Actb%%qeqwT3',
 'Actb%%tralalaT0',
 'Actb%%tralalaT1',
 'Actb%%tralalaT2',
 'Actb%%tralalaT3',
 'Dummy%%rrrrT0',
 'Dummy%%rrrrT1',
 'Dummy%%rrrrT2',
 'Dummy%%rrrrT3',
 ]

dict_samples = {'sample':[], 'gene':[], 'site':[], 'timepoint':[]}
for tp in sub_sample_list:
    gene, site, t = get_gene_site(tp)
    dict_samples['sample'].append(tp)
    dict_samples['gene'].append(str(gene))
    dict_samples[ 'site'].append(site)
    dict_samples['timepoint'].append(t) 

df = pd.DataFrame(dict_samples , 
                columns = ['sample', 'gene', 'site', 'timepoint'])

df.query('gene = "Actb"')
Contributor

jreback commented Jul 2, 2015

well, numexpr does not support this, nor are string ops actually passed. You should simply use regular indexing

df[df.gene == "Actb"]

I suppose this is a bug in that this should fall back to the python engine.

@jreback jreback added Bug Dtypes labels Jul 2, 2015

jreback added this to the Next Major Release milestone Jul 2, 2015

jreback changed the title from query: to query with invalid dtypes should fallback to python engine Jul 2, 2015

jreback changed the title from query with invalid dtypes should fallback to python engine to BUG: query with invalid dtypes should fallback to python engine Jul 2, 2015

@tsstchoi tsstchoi added a commit to tsstchoi/pandas that referenced this issue Feb 26, 2016

@tsstchoi tsstchoi Update what's new page with relevant issues for df.query() with in-pl…
…ace operator

The relevant issues added are :
BUG: query with invalid dtypes should fallback to python engine #10486
BUG: query modifies the frame when you compare with `=` #8664
414e30f

@jreback jreback modified the milestone: 0.18.2, Next Major Release May 7, 2016

jreback closed this in 881a690 May 7, 2016

@nps nps added a commit to nps/pandas that referenced this issue May 17, 2016

@tsstchoi @nps tsstchoi + nps DOC: Update what's new page with relevant issues for df.query() with …
…in-pl…


closes #10486

Author: Steve Choi <stchoi@twosigma.com>

Closes #12456 from tsstchoi/inplace_doc and squashes the following commits:

414e30f [Steve Choi] Update what's new page with relevant issues for df.query() with in-place operator
d4e9073
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment