Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError after upgrading to 1.3.13 #5110

Closed
decaz opened this issue Jan 22, 2020 · 10 comments
Closed

AttributeError after upgrading to 1.3.13 #5110

decaz opened this issue Jan 22, 2020 · 10 comments
Labels
bug Something isn't working orm regression something worked and was broken by a change
Milestone

Comments

@decaz
Copy link
Contributor

decaz commented Jan 22, 2020

I don’t know if this somehow affects the problem, but I use Flask-SQLAlchemy.

The part of code is as follows:

note = Note.query.options(noload('*')).with_for_update(of=Note).get_or_404(note_id)
db.session.refresh(note)

After upgrading SQLAlchemy version from 1.3.12 to 1.3.13 I got the following exception traceback:

  File " /home/decaz/workspace/project/views.py", line 432, in post                                                                                                                 
    db.session.refresh(note)                                                                                                                                                                                                                                                      
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/scoping.py", line 162, in do                                                                                                                                                            
    return getattr(self.registry(), name)(*args, **kwargs)                                                                       
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1675, in refresh                                                                                                                                                      
    loading.load_on_ident(                                                                                                    
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 198, in load_on_ident                                                                                  
    return load_on_pk_identity(                                                                                                                                 
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 284, in load_on_pk_identity
    return q.one()                                                                                                                        
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3347, in one                                                                                               
    ret = self.one_or_none()                                                                                                        
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3316, in one_or_none                          
    ret = list(self)                                                                                                                  
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 101, in instances         
    util.raise_from_cause(err)                                                                                            
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)                                                                                                                                                                                               
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 153, in reraise                                                                           
    raise value                                                                                                                                                                                                                           
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 81, in instances                                                              
    rows = [proc(row) for row in fetch]                                                                                                                                                                            
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 81, in <listcomp>
    rows = [proc(row) for row in fetch]                                                                                                        
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 565, in _instance                           
    _populate_full(                                                                                                                                                                                     
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 730, in _populate_full                      
    populator(state, dict_, row)                                                                                                       
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 2031, in load_collection_from_joined_existing_row                                      
    inst = _instance(row)                                                                                                                                                                             
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 565, in _instance                                                                         
    _populate_full(                                                                                                                                                                                   
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 709, in _populate_full
    elif load_path != state.load_path:
  File " /home/decaz/workspace/project/.venv/lib/python3.8/site-packages/sqlalchemy/orm/path_registry.py", line 63, in __eq__
    return other is not None and self.path == other.path
AttributeError: 'tuple' object has no attribute 'path'

@zzzeek unfortunately, right now I don't have time to prepare an example that reproduces the problem, but I hope you'll be able to understand what is the reason.

@zzzeek zzzeek added bug Something isn't working orm regression something worked and was broken by a change labels Jan 22, 2020
@zzzeek zzzeek added this to the 1.3.x milestone Jan 22, 2020
@zzzeek
Copy link
Member

zzzeek commented Jan 22, 2020

OK so I really do not want to encourage bug reports like these. the stack trace is helpful but if you can at least share the mappings, that will save me a lot of time. This is reproducible as I stepped through the stack trace and attempted to reproduce the conditions at each step and I then got a bit lucky on a few guesses. here is the test case, bisection coming next.

from sqlalchemy import Column
from sqlalchemy import create_engine
from sqlalchemy import ForeignKey
from sqlalchemy import Integer
from sqlalchemy import String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import lazyload
from sqlalchemy.orm import relationship
from sqlalchemy.orm import Session

Base = declarative_base()


class A(Base):
    __tablename__ = "a"

    id = Column(Integer, primary_key=True)
    data = Column(String)
    bs = relationship("B", lazy="joined")


class B(Base):
    __tablename__ = "b"
    id = Column(Integer, primary_key=True)
    a_id = Column(ForeignKey("a.id"))
    data = Column(String)
    c_id = Column(ForeignKey("c.id"))
    cs = relationship("C", lazy="joined")


class C(Base):
    __tablename__ = "c"
    id = Column(Integer, primary_key=True)


e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)

s = Session(e)

c1 = C()
s.add(A(bs=[B(cs=c1), B(cs=c1)]))
s.commit()
a1 = s.query(A).options(lazyload(A.bs).lazyload(B.cs)).get(1)
s.refresh(a1)


@zzzeek
Copy link
Member

zzzeek commented Jan 22, 2020

the issue does not appear in 1.4, only in 1.3. the commit that breaks it is 47f0aa4 which is the reference cycle refactor. The specific sub-commit within that breaks it is the path registry refactor in https://gerrit.sqlalchemy.org/#/c/sqlalchemy/sqlalchemy/+/1631/5..6 . The error so far seems to be that because PathRegistry does not have a __ne__() method, only an __eq__(), the SlotsEntityRegistry falls into a negation of __eq__(), which fails, but the CachingEntityRegistry which is the original version is a subclass of dict so __ne__() is going down some other path.

@zzzeek
Copy link
Member

zzzeek commented Jan 22, 2020

1.4 is not failing beacuse eager loaders are not running within the refresh() and it is not clear why this is happening. may be a new issue.

@zzzeek
Copy link
Member

zzzeek commented Jan 22, 2020

1.4 raises if we reduce the test to the below, which does run the joined loaders on the refresh:


s = Session(e)

c1 = C()
a1 = A(bs=[B(cs=c1), B(cs=c1)])
s.add(a1)
s.commit()
s.refresh(a1)

@zzzeek
Copy link
Member

zzzeek commented Jan 22, 2020

OK in 1.4, the original test case does not run the eager loaders on the refresh because of https://docs.sqlalchemy.org/en/14/changelog/migration_14.html#change-1763, where in this case the "lazyload()" option is taking effect. so this is intentional. but surprising, because this is a downgraded loader, not an upgraded one which is what the above change was intended to work with.

@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the master branch:

InstanceState default path is RootRegistry, not tuple https://gerrit.sqlalchemy.org/1677

@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the rel_1_3 branch:

InstanceState default path is RootRegistry, not tuple https://gerrit.sqlalchemy.org/1678

sqlalchemy-bot pushed a commit that referenced this issue Jan 23, 2020
Fixed regression caused in 1.3.13 by 🎫`5056` where a refactor of the
ORM path registry system made it such that a path could no longer be
compared to an empty tuple, which can occur in a particular kind of joined
eager loading path.   The "empty tuple" use case has been resolved so that
the path registry is compared to a path registry in all cases;  the
:class:`.PathRegistry` object itself now implements ``__eq__()`` and
``__ne__()`` methods which will take place for all equality comparisons and
continue to succeed in the not anticipated case that a non-
:class:`.PathRegistry` object is compared, while emitting a warning that
this object should not be the subject of the comparison.

Fixes: #5110
Change-Id: I6cab6cd771c131d12b17939b369212f12c6bee16
(cherry picked from commit f5eeac3)
@ozamani9
Copy link

Has the fix for this issue been pushed to PyPi SQLAlchemy 1.3.13? We are still seeing the error in 1.3.13 (not 1.3.12).

Traceback (most recent call last):
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/gunicorn/workers/base_async.py", line 55, in handle
self.handle_request(listener_name, req, client, addr)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/gunicorn/workers/base_async.py", line 106, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 2463, in call
return self.wsgi_app(environ, start_response)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_socketio/init.py", line 46, in call
start_response)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/engineio/middleware.py", line 74, in call
return self.wsgi_app(environ, start_response)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
response = self.handle_exception(e)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_restx/api.py", line 599, in error_router
return original_handler(f)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 1866, in handle_exception
reraise(exc_type, exc_value, tb)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_restx/api.py", line 597, in error_router
return self.handle_error(e)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_restx/api.py", line 599, in error_router
return original_handler(f)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_restx/api.py", line 597, in error_router
return self.handle_error(e)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functionsrule.endpoint
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_restx/api.py", line 339, in wrapper
resp = resource(*args, **kwargs)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask/views.py", line 89, in view
return self.dispatch_request(*args, **kwargs)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_restx/resource.py", line 44, in dispatch_request
resp = meth(*args, **kwargs)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/flask_oidc/init.py", line 860, in decorated
return view_func(*args, **kwargs)
File "/mnt/c/git/queue-management/api/qsystem.py", line 161, in decorated_function
return f(*args, **kwargs)
File "/mnt/c/git/queue-management/api/app/resources/theq/citizen/citizen_generic_invite.py", line 72, in post
db.session.refresh(citizen)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1680, in refresh
only_load_props=attribute_names,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 204, in load_on_ident
identity_token=identity_token,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 284, in load_on_pk_identity
return q.one()
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3347, in one
ret = self.one_or_none()
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3316, in one_or_none
ret = list(self)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 101, in instances
util.raise_from_cause(err)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
raise value
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 81, in instances
rows = [proc(row) for row in fetch]
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 81, in
rows = [proc(row) for row in fetch]
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 574, in _instance
populators,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 706, in populate_full
populator(state, dict
, row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 2012, in load_collection_from_joined_new_row
inst = _instance(row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 574, in _instance
populators,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 706, in populate_full
populator(state, dict
, row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 2012, in load_collection_from_joined_new_row
inst = _instance(row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 574, in instance
populators,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 706, in populate_full
populator(state, dict
, row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 2053, in load_scalar_from_joined_new_row
dict
[key] = _instance(row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 574, in instance
populators,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 706, in populate_full
populator(state, dict
, row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/strategies.py", line 2053, in load_scalar_from_joined_new_row
dict
[key] = _instance(row)
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 574, in _instance
populators,
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 709, in _populate_full
elif load_path != state.load_path:
File "/mnt/c/Git/queue-management/api/env/lib/python3.6/site-packages/sqlalchemy/orm/path_registry.py", line 63, in eq
return other is not None and self.path == other.path
AttributeError: 'tuple' object has no attribute 'path'

@zzzeek
Copy link
Member

zzzeek commented Feb 10, 2020

Has the fix for this issue been pushed to PyPi SQLAlchemy 1.3.13? We are still seeing the error in 1.3.13 (not 1.3.12).

1.3.13 is the version that the issue was reported against, it's in the title of the bug :)

the standard way to deal with a pending fix is to exclude sqlalchemy==1.3.13 in your requirements.txt file so that when 1.3.14 comes out it will upgrade to the next working version.

@ozamani9
Copy link

Thank you. I misinterpreted sqlalchemy-bot closed this in f5eeac3 19 days ago as being fixed in 1.3.13. Thank you for the clarification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working orm regression something worked and was broken by a change
Projects
None yet
Development

No branches or pull requests

4 participants