Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.7.4/0.7.5 breaks queries using the `in` operator to test membership of another query result #373

Closed
Katharine opened this Issue Aug 1, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@Katharine
Copy link

Katharine commented Aug 1, 2018

For instance, doing this will fail in 0.7.4 and 0.7.5 (and orm), but worked in 0.7 through 0.7.3:

foos = Foo.select()
Foo.select(lambda x: x in foos)

Which produces AttributeError: 'QueryType' object has no attribute '__name__'.

select(foo for foo in Foo if foo in foos) fails in the same way.

This is a complete repro:

from pony.orm import *

db = Database()


class Foo(db.Entity):
    pass


db.bind('sqlite', ':memory:', create_db=True)
db.generate_mapping(create_tables=True)


with db_session():
    foos = Foo.select()
    Foo.select(lambda x: x in foos)  # throws
    select(foo for foo in Foo if foo in foos)  # also throws

and this is the entire traceback:

Traceback (most recent call last):
  File "repro.py", line 17, in <module>
    select(foo for foo in Foo if foo in foos)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/core.py", line 5194, in select
    return make_query(args, frame_depth=cut_traceback_depth+1)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/core.py", line 5190, in make_query
    return Query(code_key, tree.code, globals, locals, cells, left_join)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/core.py", line 5347, in __init__
    translator = translator_cls(tree_copy, None, filter_num, extractors, vars, vartypes.copy(), left_join=left_join)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 338, in __init__
    translator.dispatch(if_)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 84, in dispatch
    return ASTTranslator.dispatch(translator, node)  # default route
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/asttranslation.py", line 36, in dispatch
    translator.dispatch(child)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 84, in dispatch
    return ASTTranslator.dispatch(translator, node)  # default route
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/asttranslation.py", line 31, in dispatch
    stop = translator.call(pre_method, node)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 140, in call
    reraise(exc_class, exc, tb)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/utils/utils.py", line 95, in reraise
    try: raise exc.with_traceback(tb)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 130, in call
    try: monad = method(translator, node)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 859, in preCompare
    translator.dispatch(right)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 85, in dispatch
    translator.call(translator.__class__.dispatch_external, node)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 140, in call
    reraise(exc_class, exc, tb)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/utils/utils.py", line 95, in reraise
    try: raise exc.with_traceback(tb)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 130, in call
    try: monad = method(translator, node)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 124, in dispatch_external
    monad = ParamMonad.new(translator, t, (varkey, None, None))
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 2036, in new
    type = normalize_type(type)
  File "/home/ktbry/.local/share/virtualenvs/pony-bug-repro-9j2oy5zW/lib/python3.5/site-packages/pony/orm/ormtypes.py", line 187, in normalize_type
    assert t.__name__ != 'EntityMeta'
AttributeError: 'QueryType' object has no attribute '__name__'
@benmod

This comment has been minimized.

Copy link

benmod commented Aug 2, 2018

I am encountering the same issue.

Here is our traceback if it helps:

web_1         |     return select(I for I in db.Issue if I.affectedClient in clients)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 5194, in select
web_1         |     return make_query(args, frame_depth=cut_traceback_depth+1)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 5190, in make_query
web_1         |     return Query(code_key, tree.code, globals, locals, cells, left_join)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 5347, in __init__
web_1         |     translator = translator_cls(tree_copy, None, filter_num, extractors, vars, vartypes.copy(), left_join=left_join)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 338, in __init__
web_1         |     translator.dispatch(if_)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 84, in dispatch
web_1         |     return ASTTranslator.dispatch(translator, node)  # default route
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/asttranslation.py", line 36, in dispatch
web_1         |     translator.dispatch(child)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 84, in dispatch
web_1         |     return ASTTranslator.dispatch(translator, node)  # default route
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/asttranslation.py", line 31, in dispatch
web_1         |     stop = translator.call(pre_method, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 140, in call
web_1         |     reraise(exc_class, exc, tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/utils/utils.py", line 95, in reraise
web_1         |     try: raise exc.with_traceback(tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 130, in call
web_1         |     try: monad = method(translator, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 859, in preCompare
web_1         |     translator.dispatch(right)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 85, in dispatch
web_1         |     translator.call(translator.__class__.dispatch_external, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 140, in call
web_1         |     reraise(exc_class, exc, tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/utils/utils.py", line 95, in reraise
web_1         |     try: raise exc.with_traceback(tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 130, in call
web_1         |     try: monad = method(translator, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 124, in dispatch_external
web_1         |     monad = ParamMonad.new(translator, t, (varkey, None, None))
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 2036, in new
web_1         |     type = normalize_type(type)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/ormtypes.py", line 187, in normalize_type
web_1         |     assert t.__name__ != 'EntityMeta'
web_1         | AttributeError: 'QueryType' object has no attribute '__name__'
web_1         |
web_1         | During handling of the above exception, another exception occurred:
web_1         |
web_1         | Traceback (most recent call last):
web_1         |   File "/usr/local/lib/python3.5/site-packages/gunicorn/workers/sync.py", line 135, in handle
web_1         |     self.handle_request(listener, req, client, addr)
web_1         |   File "/usr/local/lib/python3.5/site-packages/gunicorn/workers/sync.py", line 176, in handle_request
web_1         |     respiter = self.wsgi(environ, resp.start_response)
web_1         |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 2309, in __call__
web_1         |     return self.wsgi_app(environ, start_response)
web_1         |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 2295, in wsgi_app
web_1         |     response = self.handle_exception(e)
web_1         |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 1748, in handle_exception
web_1         |     return self.finalize_request(handler(e), from_error_handler=True)
web_1         |   File "<string>", line 2, in internal_error
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 466, in new_func
web_1         |     result = func(*args, **kwargs)
web_1         |   File "/crm/web_interface/error_handler.py", line 51, in internal_error
web_1         |     return render_template('macros/500.html', Error=ErrorLog), 500
web_1         |   File "/usr/local/lib/python3.5/site-packages/flask/templating.py", line 133, in render_template
web_1         |     ctx.app.update_template_context(context)
web_1         |   File "/usr/local/lib/python3.5/site-packages/flask/app.py", line 792, in update_template_context
web_1         |     context.update(func())
web_1         |   File "<string>", line 2, in notification_processor
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 466, in new_func
web_1         |     result = func(*args, **kwargs)
web_1         |   File "/crm/web_interface/context.py", line 196, in notification_processor
web_1         |     IssuesAwaiting = current_user.getAllOpenIssues().count()
web_1         |   File "/crm/client_user_system/model/user_model.py", line 286, in getAllOpenIssues
web_1         |     issues = self.getAllIssues()
web_1         |   File "/crm/client_user_system/model/user_model.py", line 282, in getAllIssues
web_1         |     issues = self.getIssuesFromAffectedClients(clients)
web_1         |   File "/crm/client_user_system/model/user_model.py", line 278, in getIssuesFromAffectedClients
web_1         |     return select(I for I in db.Issue if I.affectedClient in clients)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 5194, in select
web_1         |     return make_query(args, frame_depth=cut_traceback_depth+1)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 5190, in make_query
web_1         |     return Query(code_key, tree.code, globals, locals, cells, left_join)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/core.py", line 5347, in __init__
web_1         |     translator = translator_cls(tree_copy, None, filter_num, extractors, vars, vartypes.copy(), left_join=left_join)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 338, in __init__
web_1         |     translator.dispatch(if_)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 84, in dispatch
web_1         |     return ASTTranslator.dispatch(translator, node)  # default route
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/asttranslation.py", line 36, in dispatch
web_1         |     translator.dispatch(child)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 84, in dispatch
web_1         |     return ASTTranslator.dispatch(translator, node)  # default route
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/asttranslation.py", line 31, in dispatch
web_1         |     stop = translator.call(pre_method, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 140, in call
web_1         |     reraise(exc_class, exc, tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/utils/utils.py", line 95, in reraise
web_1         |     try: raise exc.with_traceback(tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 130, in call
web_1         |     try: monad = method(translator, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 859, in preCompare
web_1         |     translator.dispatch(right)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 85, in dispatch
web_1         |     translator.call(translator.__class__.dispatch_external, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 140, in call
web_1         |     reraise(exc_class, exc, tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/utils/utils.py", line 95, in reraise
web_1         |     try: raise exc.with_traceback(tb)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 130, in call
web_1         |     try: monad = method(translator, node)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 124, in dispatch_external
web_1         |     monad = ParamMonad.new(translator, t, (varkey, None, None))
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/sqltranslation.py", line 2036, in new
web_1         |     type = normalize_type(type)
web_1         |   File "/usr/local/lib/python3.5/site-packages/pony/orm/ormtypes.py", line 187, in normalize_type
web_1         |     assert t.__name__ != 'EntityMeta'
web_1         | AttributeError: 'QueryType' object has no attribute '__name__'

@kozlovsky kozlovsky closed this in fe6a9fd Aug 2, 2018

@kozlovsky

This comment has been minimized.

Copy link
Member

kozlovsky commented Aug 2, 2018

Should work now!

Previously such conditions were translated as

and column in (?, ?, ?, ..., ?)

where (?, ?, ?, ..., ?) were ids fetched by previous query

Now it translates to:

and column in (select ...)

where previous query is inlined to a new query.

If you want to keep previous behavior, you can force fetching of first query by using foos[:] instead of foos

kozlovsky added a commit that referenced this issue Aug 8, 2018

Pony ORM Release 0.7.6rc1 (2018-08-08)
# New features

* f-strings support in queries: select(f'{s.name} - {s.age}' for s in Student)
* #344: It is now possible to specify offset without limit: `query.limit(offset=10)`
* #371: Support of explicit casting of JSON expressions to `str`, `int` or `float`
* `@db.on_connect` decorator added

# Bugfixes

* Fix bulk delete bug introduced in 0.7.4
* #370 Fix memory leak introduced in 0.7.4
* Now exists() in query does not throw away condition in generator expression: `exists(s.gpa > 3 for s in Student)`
* #373: 0.7.4/0.7.5 breaks queries using the `in` operator to test membership of another query result
* #374: `auto=True` can be used with all PrimaryKey types, not only int
* #369: Make QueryResult looks like a list object again: add concatenation with lists, `.shuffle()` and `.to_list()` methods
* #355: Fix binary primary keys `PrimaryKey(buffer)` in Python2
* Interactive mode support for PyCharm console
* Fix wrong table aliases in complex queries
* Fix query optimization code for complex queries

kozlovsky added a commit that referenced this issue Aug 10, 2018

Pony ORM Release 0.7.6 (2018-08-10)
# Features since 0.7.5:

* f-strings support in queries: `select(f'{s.name} - {s.age}' for s in Student)`
* #344: It is now possible to specify offset without limit: `query.limit(offset=10)`
* #371: Support of explicit casting of JSON expressions to `str`, `int` or `float`
* `@db.on_connect` decorator added

# Bugfixes

* Fix bulk delete bug introduced in 0.7.4
* #370 Fix memory leak introduced in 0.7.4
* Now `exists()` in query does not throw away condition in generator expression: `exists(s.gpa > 3 for s in Student)`
* #373: 0.7.4/0.7.5 breaks queries using the `in` operator to test membership of another query result
* #374: `auto=True` can be used with all PrimaryKey types, not only `int`
* #369: Make QueryResult looks like a list object again: add concatenation with lists, `.shuffle()` and `.to_list()` methods
* #355: Fix binary primary keys `PrimaryKey(buffer)` in Python2
* Interactive mode support for PyCharm console
* Fix wrong table aliases in complex queries
* Fix query optimization code for complex queries
* Fix a bug with hybrid properties that use external functions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.