Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support column names with special characters in mapd #1675

Closed
wants to merge 5 commits into from

Conversation

Projects
2 participants
@saulshanabrook
Copy link
Contributor

commented Nov 19, 2018

The mapd backend raises an error currently when you include some special characters in new table names

Ex:

conn = ibis.mapd.connect(
    host='metis.mapd.com', user='mapd', password='HyperInteractive',
    port=443, database='mapd', protocol= 'https'
)
t = conn.table("flights_donotmodify")
t.aggregate(t.count().name("count_*")).execute()

Gives error

---------------------------------------------------------------------------
TMapDException                            Traceback (most recent call last)
/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/pymapd/cursor.py in execute(self, operation, parameters)
    119                 column_format=self.columnar,
--> 120                 nonce=None, first_n=-1, at_most_n=-1)
    121         except T.TMapDException as e:

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/mapd/MapD.py in sql_execute(self, session, query, column_format, nonce, first_n, at_most_n)
   1457         self.send_sql_execute(session, query, column_format, nonce, first_n, at_most_n)
-> 1458         return self.recv_sql_execute()
   1459 

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/mapd/MapD.py in recv_sql_execute(self)
   1486         if result.e is not None:
-> 1487             raise result.e
   1488         raise TApplicationException(TApplicationException.MISSING_RESULT, "sql_execute failed: unknown result")

TMapDException: TMapDException(error_msg='Exception: Parse failed: Encountered "*" at line 1, column 26.\nWas expecting one of:\n    <EOF> \n    "ORDER" ...\n    "LIMIT" ...\n    "OFFSET" ...\n    "FETCH" ...\n    "FROM" ...\n    "," ...\n    "UNION" ...\n    "INTERSECT" ...\n    "EXCEPT" ...\n    "MINUS" ...\n    ')

The above exception was the direct cause of the following exception:

ProgrammingError                          Traceback (most recent call last)
/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/mapd/client.py in _execute(self, query, results)
    407         try:
--> 408             result = MapDCursor(execute(query))
    409         except Exception as e:

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/pymapd/cursor.py in execute(self, operation, parameters)
    121         except T.TMapDException as e:
--> 122             six.raise_from(_translate_exception(e), e)
    123         self._description = _extract_description(result.row_set.row_desc)

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/six.py in raise_from(value, from_value)

ProgrammingError: Exception: Parse failed: Encountered "*" at line 1, column 26.
Was expecting one of:
    <EOF> 
    "ORDER" ...
    "LIMIT" ...
    "OFFSET" ...
    "FETCH" ...
    "FROM" ...
    "," ...
    "UNION" ...
    "INTERSECT" ...
    "EXCEPT" ...
    "MINUS" ...
    

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
<ipython-input-12-ca2aa7d96001> in <module>
----> 1 t.aggregate(t.count().name("count_*")).execute()

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/expr/types.py in execute(self, limit, params, **kwargs)
    195         """
    196         from ibis.client import execute
--> 197         return execute(self, limit=limit, params=params, **kwargs)
    198 
    199     def compile(self, limit=None, params=None):

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/client.py in execute(expr, limit, params, **kwargs)
    282 def execute(expr, limit='default', params=None, **kwargs):
    283     backend, = validate_backends(list(find_backends(expr)))
--> 284     return backend.execute(expr, limit=limit, params=params, **kwargs)
    285 
    286 

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/client.py in execute(self, expr, params, limit, **kwargs)
    182         """
    183         query_ast = self._build_ast_ensure_limit(expr, limit, params=params)
--> 184         result = self._execute_query(query_ast, **kwargs)
    185         return result
    186 

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/client.py in _execute_query(self, dml, **kwargs)
    187     def _execute_query(self, dml, **kwargs):
    188         query = self.query_class(self, dml, **kwargs)
--> 189         return query.execute()
    190 
    191     def compile(self, expr, params=None, limit=None):

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/client.py in execute(self)
     39     def execute(self):
     40         # synchronous by default
---> 41         with self.client._execute(self.compiled_sql, results=True) as cur:
     42             result = self._fetch(cur)
     43 

/usr/local/miniconda3/envs/jupyterlab-omnisci/lib/python3.6/site-packages/ibis/mapd/client.py in _execute(self, query, results)
    408             result = MapDCursor(execute(query))
    409         except Exception as e:
--> 410             raise Exception('{}: {}'.format(e, query))
    411 
    412         if results:

Exception: Exception: Parse failed: Encountered "*" at line 1, column 26.
Was expecting one of:
    <EOF> 
    "ORDER" ...
    "LIMIT" ...
    "OFFSET" ...
    "FETCH" ...
    "FROM" ...
    "," ...
    "UNION" ...
    "INTERSECT" ...
    "EXCEPT" ...
    "MINUS" ...
    : SELECT count(*) AS count_*
FROM flights_donotmodify
LIMIT 10000

If instead we quote the name, then the query succeeds (like this):

SELECT count(*) AS "count_*"
FROM flights_donotmodify
LIMIT 10000

This PR will fix this by quoting names with * in them.

  • Add failing test case
  • fix test case

@saulshanabrook saulshanabrook changed the title WIP: Support column names with special characters in mapd Support column names with special characters in mapd Nov 19, 2018

@codecov

This comment has been minimized.

Copy link

commented Nov 19, 2018

Codecov Report

Merging #1675 into master will decrease coverage by 2.63%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1675      +/-   ##
==========================================
- Coverage   90.05%   87.42%   -2.64%     
==========================================
  Files         186      186              
  Lines       27263    27265       +2     
  Branches     2311     2310       -1     
==========================================
- Hits        24553    23836     -717     
- Misses       2308     3023     +715     
- Partials      402      406       +4
Impacted Files Coverage Δ
ibis/mapd/operations.py 71.29% <100%> (-0.19%) ⬇️
ibis/mapd/identifiers.py 100% <100%> (+22.22%) ⬆️
ibis/mapd/tests/test_operations.py 97.36% <100%> (+0.22%) ⬆️
ibis/bigquery/tests/test_client.py 25.87% <0%> (-73.55%) ⬇️
ibis/bigquery/tests/test_compiler.py 39.57% <0%> (-60.43%) ⬇️
ibis/bigquery/udf/tests/test_udf_execute.py 28.71% <0%> (-53.47%) ⬇️
ibis/bigquery/client.py 42.67% <0%> (-52.59%) ⬇️
ibis/bigquery/compiler.py 58.67% <0%> (-38.75%) ⬇️
ibis/bigquery/tests/conftest.py 70.27% <0%> (-16.22%) ⬇️
ibis/bigquery/udf/api.py 80.48% <0%> (-14.64%) ⬇️
... and 13 more

@cpcloud cpcloud self-assigned this Nov 21, 2018

@cpcloud cpcloud added this to In progress in MapD via automation Nov 21, 2018

@cpcloud cpcloud added this to the 1.0.0 milestone Nov 21, 2018

MapD automation moved this from In progress to Needs review Nov 21, 2018

@@ -92,3 +92,13 @@ def test_where_operator(alltypes):
counts = expr.execute().value_counts()
assert counts[0] == 5
assert counts[1] == 5


@pytest.mark.parametrize(('name',), [

This comment has been minimized.

Copy link
@cpcloud

cpcloud Nov 21, 2018

Member

('name',) doesn't need to be a tuple. You can write this as

@pytest.mark.parametrize(
    'name', ['regular_name', 'star_name', 'space_name']
)

This comment has been minimized.

Copy link
@cpcloud

cpcloud Nov 21, 2018

Member

Also, this test isn't testing the code you added, since it's just renaming the result of a table count operation to be a vanilla column name. Please add a test that uses column names with stars, spaces, and other characters. You could even parameterize over string.punctuation.

This comment has been minimized.

Copy link
@saulshanabrook

saulshanabrook Nov 21, 2018

Author Contributor

Thanks for the tuple suggestions, just updated.

I believe it is testing the code I added. If I revert back the changes in ibis/mapd/operations.py, the tests will fail:

bash-3.2$ py.test ibis/mapd/tests/test_operations.py::test_quote_name
========================================================= test session starts =========================================================
platform darwin -- Python 3.6.6, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/saul/p/ibis, inifile:
plugins: xdist-1.24.1, forked-0.2
collected 3 items

ibis/mapd/tests/test_operations.py FFF                                                                                          [100%]

============================================================== FAILURES ===============================================================
____________________________________________________ test_quote_name[regular_name] ____________________________________________________

alltypes = DatabaseTable[table]
  name: functional_alltypes
  schema:
    index : int64
    Unnamed__0 : int64
    id : int32
   ...    date_string_col : string
    string_col : string
    timestamp_col : timestamp
    year_ : int32
    month_ : int32
name = 'regular_name'

    @pytest.mark.parametrize(('name',), [
        ('regular_name',),
        ('star_name*',),
        ('space_name ',),
    ])
    def test_quote_name(alltypes, name):
        expr = alltypes.aggregate(alltypes.count().name(name))
>       assert name in expr.execute()

ibis/mapd/tests/test_operations.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ibis/expr/types.py:197: in execute
    return execute(self, limit=limit, params=params, **kwargs)
ibis/client.py:284: in execute
    return backend.execute(expr, limit=limit, params=params, **kwargs)
ibis/client.py:184: in execute
    result = self._execute_query(query_ast, **kwargs)
ibis/client.py:188: in _execute_query
    query = self.query_class(self, dml, **kwargs)
ibis/client.py:32: in __init__
    self.compiled_sql = sql.compile()
ibis/sql/compiler.py:54: in compile
    compiled_queries = [q.compile() for q in self.queries]
ibis/sql/compiler.py:54: in <listcomp>
    compiled_queries = [q.compile() for q in self.queries]
ibis/sql/compiler.py:1514: in compile
    select_frag = self.format_select_set()
ibis/mapd/compiler.py:81: in format_select_set
    return super().format_select_set()
ibis/sql/compiler.py:1567: in format_select_set
    expr_str = self._translate(expr, named=True)
ibis/sql/compiler.py:1469: in _translate
    return translator.get_result()
ibis/sql/compiler.py:1247: in get_result
    translated = self.name(translated, name)
ibis/mapd/compiler.py:200: in name
    return mapd_ops._name_expr(translated, name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

formatted_expr = 'count(*)', quoted_name = 'regular_name'

    def _name_expr(formatted_expr, quoted_name):
>       if quoted_name in _identifiers:
E       NameError: name '_identifiers' is not defined

ibis/mapd/operations.py:384: NameError
_____________________________________________________ test_quote_name[star_name*] _____________________________________________________

alltypes = DatabaseTable[table]
  name: functional_alltypes
  schema:
    index : int64
    Unnamed__0 : int64
    id : int32
   ...    date_string_col : string
    string_col : string
    timestamp_col : timestamp
    year_ : int32
    month_ : int32
name = 'star_name*'

    @pytest.mark.parametrize(('name',), [
        ('regular_name',),
        ('star_name*',),
        ('space_name ',),
    ])
    def test_quote_name(alltypes, name):
        expr = alltypes.aggregate(alltypes.count().name(name))
>       assert name in expr.execute()

ibis/mapd/tests/test_operations.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ibis/expr/types.py:197: in execute
    return execute(self, limit=limit, params=params, **kwargs)
ibis/client.py:284: in execute
    return backend.execute(expr, limit=limit, params=params, **kwargs)
ibis/client.py:184: in execute
    result = self._execute_query(query_ast, **kwargs)
ibis/client.py:188: in _execute_query
    query = self.query_class(self, dml, **kwargs)
ibis/client.py:32: in __init__
    self.compiled_sql = sql.compile()
ibis/sql/compiler.py:54: in compile
    compiled_queries = [q.compile() for q in self.queries]
ibis/sql/compiler.py:54: in <listcomp>
    compiled_queries = [q.compile() for q in self.queries]
ibis/sql/compiler.py:1514: in compile
    select_frag = self.format_select_set()
ibis/mapd/compiler.py:81: in format_select_set
    return super().format_select_set()
ibis/sql/compiler.py:1567: in format_select_set
    expr_str = self._translate(expr, named=True)
ibis/sql/compiler.py:1469: in _translate
    return translator.get_result()
ibis/sql/compiler.py:1247: in get_result
    translated = self.name(translated, name)
ibis/mapd/compiler.py:200: in name
    return mapd_ops._name_expr(translated, name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

formatted_expr = 'count(*)', quoted_name = 'star_name*'

    def _name_expr(formatted_expr, quoted_name):
>       if quoted_name in _identifiers:
E       NameError: name '_identifiers' is not defined

ibis/mapd/operations.py:384: NameError
____________________________________________________ test_quote_name[space_name ] _____________________________________________________

alltypes = DatabaseTable[table]
  name: functional_alltypes
  schema:
    index : int64
    Unnamed__0 : int64
    id : int32
   ...    date_string_col : string
    string_col : string
    timestamp_col : timestamp
    year_ : int32
    month_ : int32
name = 'space_name '

    @pytest.mark.parametrize(('name',), [
        ('regular_name',),
        ('star_name*',),
        ('space_name ',),
    ])
    def test_quote_name(alltypes, name):
        expr = alltypes.aggregate(alltypes.count().name(name))
>       assert name in expr.execute()

ibis/mapd/tests/test_operations.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ibis/expr/types.py:197: in execute
    return execute(self, limit=limit, params=params, **kwargs)
ibis/client.py:284: in execute
    return backend.execute(expr, limit=limit, params=params, **kwargs)
ibis/client.py:184: in execute
    result = self._execute_query(query_ast, **kwargs)
ibis/client.py:188: in _execute_query
    query = self.query_class(self, dml, **kwargs)
ibis/client.py:32: in __init__
    self.compiled_sql = sql.compile()
ibis/sql/compiler.py:54: in compile
    compiled_queries = [q.compile() for q in self.queries]
ibis/sql/compiler.py:54: in <listcomp>
    compiled_queries = [q.compile() for q in self.queries]
ibis/sql/compiler.py:1514: in compile
    select_frag = self.format_select_set()
ibis/mapd/compiler.py:81: in format_select_set
    return super().format_select_set()
ibis/sql/compiler.py:1567: in format_select_set
    expr_str = self._translate(expr, named=True)
ibis/sql/compiler.py:1469: in _translate
    return translator.get_result()
ibis/sql/compiler.py:1247: in get_result
    translated = self.name(translated, name)
ibis/mapd/compiler.py:200: in name
    return mapd_ops._name_expr(translated, name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

formatted_expr = 'count(*)', quoted_name = 'space_name '

    def _name_expr(formatted_expr, quoted_name):
>       if quoted_name in _identifiers:
E       NameError: name '_identifiers' is not defined

ibis/mapd/operations.py:384: NameError
====================================================== 3 failed in 1.36 seconds =======================================================
bash-3.2$ py.test ibis/mapd/tests/test_operations.py::test_quote_name
========================================================= test session starts =========================================================
platform darwin -- Python 3.6.6, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/saul/p/ibis, inifile:
plugins: xdist-1.24.1, forked-0.2
collected 3 items

ibis/mapd/tests/test_operations.py .FF                                                                                          [100%]

============================================================== FAILURES ===============================================================
_____________________________________________________ test_quote_name[star_name*] _____________________________________________________

self = <pymapd.cursor.Cursor object at 0x11956b9b0>, operation = 'SELECT count(*) AS star_name*\nFROM functional_alltypes\nLIMIT 10000'
parameters = None

    def execute(self, operation, parameters=None):
        # type: (str, tuple) -> Cursor
        """Execute a SQL statement.

        Parameters
        ----------
        operation : str
            A SQL query
        parameters : dict
            Parameters to substitute into ``operation``.

        Returns
        -------
        self : Cursor

        Examples
        --------
        >>> c = conn.cursor()
        >>> c.execute("select symbol, qty from stocks")
        >>> list(c)
        [('RHAT', 100.0), ('IBM', 1000.0), ('MSFT', 1000.0), ('IBM', 500.0)]

        Passing in ``parameters``:

        >>> c.execute("select symbol qty from stocks where qty <= :max_qty",
        ...           parameters={"max_qty": 500})
        [('RHAT', 100.0), ('IBM', 500.0)]
        """
        if parameters is not None:
            operation = six.text_type(_bind_parameters(operation, parameters))
        self.rowcount = -1
        try:
            result = self.connection._client.sql_execute(
                self.connection._session, operation,
                column_format=self.columnar,
>               nonce=None, first_n=-1, at_most_n=-1)

/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pymapd/cursor.py:120:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <mapd.MapD.Client object at 0x119548c88>, session = 'KJFvqUyXHftlQTu8nHpXXGEUtmY1omu4'
query = 'SELECT count(*) AS star_name*\nFROM functional_alltypes\nLIMIT 10000', column_format = True, nonce = None, first_n = -1
at_most_n = -1

    def sql_execute(self, session, query, column_format, nonce, first_n, at_most_n):
        """
        Parameters:
         - session
         - query
         - column_format
         - nonce
         - first_n
         - at_most_n
        """
        self.send_sql_execute(session, query, column_format, nonce, first_n, at_most_n)
>       return self.recv_sql_execute()

/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/mapd/MapD.py:1458:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <mapd.MapD.Client object at 0x119548c88>

    def recv_sql_execute(self):
        iprot = self._iprot
        (fname, mtype, rseqid) = iprot.readMessageBegin()
        if mtype == TMessageType.EXCEPTION:
            x = TApplicationException()
            x.read(iprot)
            iprot.readMessageEnd()
            raise x
        result = sql_execute_result()
        result.read(iprot)
        iprot.readMessageEnd()
        if result.success is not None:
            return result.success
        if result.e is not None:
>           raise result.e
E           mapd.ttypes.TMapDException: TMapDException(error_msg='Exception: Parse failed: Encountered "*" at line 1, column 29.\nWas expecting one of:\n    <EOF> \n    "ORDER" ...\n    "LIMIT" ...\n    "OFFSET" ...\n    "FETCH" ...\n    "FROM" ...\n    "," ...\n    "UNION" ...\n    "INTERSECT" ...\n    "EXCEPT" ...\n    "MINUS" ...\n    ')

/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/mapd/MapD.py:1487: TMapDException

The above exception was the direct cause of the following exception:

self = <ibis.mapd.client.MapDClient object at 0x119548cc0>
query = 'SELECT count(*) AS star_name*\nFROM functional_alltypes\nLIMIT 10000', results = True

    def _execute(self, query, results=True):
        """

        query:
        :return:
        """
        if isinstance(query, (DDL, DML)):
            query = query.compile()

        if self.execution_type == EXECUTION_TYPE_ICP:
            execute = self.con.select_ipc
        elif self.execution_type == EXECUTION_TYPE_ICP_GPU:
            execute = self.con.select_ipc_gpu
        else:
            execute = self.con.cursor().execute

        try:
>           result = MapDCursor(execute(query))

ibis/mapd/client.py:408:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pymapd.cursor.Cursor object at 0x11956b9b0>, operation = 'SELECT count(*) AS star_name*\nFROM functional_alltypes\nLIMIT 10000'
parameters = None

    def execute(self, operation, parameters=None):
        # type: (str, tuple) -> Cursor
        """Execute a SQL statement.

        Parameters
        ----------
        operation : str
            A SQL query
        parameters : dict
            Parameters to substitute into ``operation``.

        Returns
        -------
        self : Cursor

        Examples
        --------
        >>> c = conn.cursor()
        >>> c.execute("select symbol, qty from stocks")
        >>> list(c)
        [('RHAT', 100.0), ('IBM', 1000.0), ('MSFT', 1000.0), ('IBM', 500.0)]

        Passing in ``parameters``:

        >>> c.execute("select symbol qty from stocks where qty <= :max_qty",
        ...           parameters={"max_qty": 500})
        [('RHAT', 100.0), ('IBM', 500.0)]
        """
        if parameters is not None:
            operation = six.text_type(_bind_parameters(operation, parameters))
        self.rowcount = -1
        try:
            result = self.connection._client.sql_execute(
                self.connection._session, operation,
                column_format=self.columnar,
                nonce=None, first_n=-1, at_most_n=-1)
        except T.TMapDException as e:
>           six.raise_from(_translate_exception(e), e)

/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pymapd/cursor.py:122:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

value = None
from_value = TMapDException(error_msg='Exception: Parse failed: Encountered "*" at line 1, column 29.\nWas expecting one of:\n    <...ETCH" ...\n    "FROM" ...\n    "," ...\n    "UNION" ...\n    "INTERSECT" ...\n    "EXCEPT" ...\n    "MINUS" ...\n    ')

>   ???
E   pymapd.exceptions.ProgrammingError: Exception: Parse failed: Encountered "*" at line 1, column 29.
E   Was expecting one of:
E       <EOF>
E       "ORDER" ...
E       "LIMIT" ...
E       "OFFSET" ...
E       "FETCH" ...
E       "FROM" ...
E       "," ...
E       "UNION" ...
E       "INTERSECT" ...
E       "EXCEPT" ...
E       "MINUS" ...

<string>:3: ProgrammingError

During handling of the above exception, another exception occurred:

alltypes = DatabaseTable[table]
  name: functional_alltypes
  schema:
    index : int64
    Unnamed__0 : int64
    id : int32
   ...    date_string_col : string
    string_col : string
    timestamp_col : timestamp
    year_ : int32
    month_ : int32
name = 'star_name*'

    @pytest.mark.parametrize(('name',), [
        ('regular_name',),
        ('star_name*',),
        ('space_name ',),
    ])
    def test_quote_name(alltypes, name):
        expr = alltypes.aggregate(alltypes.count().name(name))
>       assert name in expr.execute()

ibis/mapd/tests/test_operations.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ibis/expr/types.py:197: in execute
    return execute(self, limit=limit, params=params, **kwargs)
ibis/client.py:284: in execute
    return backend.execute(expr, limit=limit, params=params, **kwargs)
ibis/client.py:184: in execute
    result = self._execute_query(query_ast, **kwargs)
ibis/client.py:189: in _execute_query
    return query.execute()
ibis/client.py:41: in execute
    with self.client._execute(self.compiled_sql, results=True) as cur:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ibis.mapd.client.MapDClient object at 0x119548cc0>
query = 'SELECT count(*) AS star_name*\nFROM functional_alltypes\nLIMIT 10000', results = True

    def _execute(self, query, results=True):
        """

        query:
        :return:
        """
        if isinstance(query, (DDL, DML)):
            query = query.compile()

        if self.execution_type == EXECUTION_TYPE_ICP:
            execute = self.con.select_ipc
        elif self.execution_type == EXECUTION_TYPE_ICP_GPU:
            execute = self.con.select_ipc_gpu
        else:
            execute = self.con.cursor().execute

        try:
            result = MapDCursor(execute(query))
        except Exception as e:
>           raise Exception('{}: {}'.format(e, query))
E           Exception: Exception: Parse failed: Encountered "*" at line 1, column 29.
E           Was expecting one of:
E               <EOF>
E               "ORDER" ...
E               "LIMIT" ...
E               "OFFSET" ...
E               "FETCH" ...
E               "FROM" ...
E               "," ...
E               "UNION" ...
E               "INTERSECT" ...
E               "EXCEPT" ...
E               "MINUS" ...
E               : SELECT count(*) AS star_name*
E           FROM functional_alltypes
E           LIMIT 10000

ibis/mapd/client.py:410: Exception
____________________________________________________ test_quote_name[space_name ] _____________________________________________________

self = Index(['space_name'], dtype='object'), key = 'space_name ', method = None, tolerance = None

    @Appender(_index_shared_docs['get_loc'])
    def get_loc(self, key, method=None, tolerance=None):
        if method is None:
            if tolerance is not None:
                raise ValueError('tolerance argument only valid if using pad, '
                                 'backfill or nearest lookups')
            try:
>               return self._engine.get_loc(key)

/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pandas/core/indexes/base.py:3078:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

pandas/_libs/index.pyx:140:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

pandas/_libs/index.pyx:162:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???

pandas/_libs/hashtable_class_helper.pxi:1492:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   KeyError: 'space_name '

pandas/_libs/hashtable_class_helper.pxi:1500: KeyError

During handling of the above exception, another exception occurred:

alltypes = DatabaseTable[table]
  name: functional_alltypes
  schema:
    index : int64
    Unnamed__0 : int64
    id : int32
   ...    date_string_col : string
    string_col : string
    timestamp_col : timestamp
    year_ : int32
    month_ : int32
name = 'space_name '

    @pytest.mark.parametrize(('name',), [
        ('regular_name',),
        ('star_name*',),
        ('space_name ',),
    ])
    def test_quote_name(alltypes, name):
        expr = alltypes.aggregate(alltypes.count().name(name))
>       assert name in expr.execute()

ibis/mapd/tests/test_operations.py:104:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
ibis/expr/types.py:197: in execute
    return execute(self, limit=limit, params=params, **kwargs)
ibis/client.py:284: in execute
    return backend.execute(expr, limit=limit, params=params, **kwargs)
ibis/client.py:184: in execute
    result = self._execute_query(query_ast, **kwargs)
ibis/client.py:189: in _execute_query
    return query.execute()
ibis/client.py:42: in execute
    result = self._fetch(cur)
ibis/mapd/client.py:148: in _fetch
    return self.schema().apply_to(cursor.to_df())
ibis/pandas/client.py:324: in ibis_schema_apply_to
    col = df[column]
/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pandas/core/frame.py:2688: in __getitem__
    return self._getitem_column(key)
/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pandas/core/frame.py:2695: in _getitem_column
    return self._get_item_cache(key)
/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pandas/core/generic.py:2489: in _get_item_cache
    values = self._data.get(item)
/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pandas/core/internals.py:4115: in get
    loc = self.items.get_loc(item)
/usr/local/miniconda3/envs/ibis/lib/python3.6/site-packages/pandas/core/indexes/base.py:3080: in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
pandas/_libs/index.pyx:140: in pandas._libs.index.IndexEngine.get_loc
    ???
pandas/_libs/index.pyx:162: in pandas._libs.index.IndexEngine.get_loc
    ???
pandas/_libs/hashtable_class_helper.pxi:1492: in pandas._libs.hashtable.PyObjectHashTable.get_item
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   KeyError: 'space_name '

pandas/_libs/hashtable_class_helper.pxi:1500: KeyError
================================================= 2 failed, 1 passed in 2.67 seconds ==================================================

This comment has been minimized.

Copy link
@cpcloud

cpcloud Nov 25, 2018

Member

Yep, you're right. I missed the characters at the end of the column names.

MapD automation moved this from Needs review to Reviewer approved Nov 25, 2018

@cpcloud
Copy link
Member

left a comment

Thanks! Merging.

@cpcloud cpcloud closed this in 5672feb Nov 25, 2018

MapD automation moved this from Reviewer approved to Done Nov 25, 2018

@saulshanabrook saulshanabrook deleted the Quansight:mapd-quote-name branch Nov 25, 2018

@saulshanabrook

This comment has been minimized.

Copy link
Contributor Author

commented Nov 25, 2018

Sweet, thank you for reviewing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.