Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'NoneType' object has no attribute 'seeds' #314

Closed
NiklasRosenstein opened this issue Nov 22, 2017 · 3 comments
Closed

AttributeError: 'NoneType' object has no attribute 'seeds' #314

NiklasRosenstein opened this issue Nov 22, 2017 · 3 comments
Assignees
Labels
Milestone

Comments

@NiklasRosenstein
Copy link

I've been able to create a small example to reproduce this issue. It seems to be a combination of using an Entity subclass and accessing member functions.

Error

Traceback (most recent call last):
  File "test.py", line 64, in <module>
    print(list(list_location(['com.example'])))
  File "test.py", line 48, in <genexpr>
    return (x.as_tuple() for x in Location.get_by_path(path).children)
  File "test.py", line 35, in as_tuple
    if not self.parent:
  File "<string>", line 2, in __get__
  File "C:\Users\niklas\.virtualenvs\fatartifacts-LoWBpE4v\lib\site-packages\pony\utils\utils.py", line 58, in cut_traceback
    return func(*args, **kwargs)
  File "C:\Users\niklas\.virtualenvs\fatartifacts-LoWBpE4v\lib\site-packages\pony\orm\core.py", line 2099, in __get__
    value = attr.get(obj)
  File "C:\Users\niklas\.virtualenvs\fatartifacts-LoWBpE4v\lib\site-packages\pony\orm\core.py", line 2111, in get
    seeds = obj._session_cache_.seeds[val._pk_attrs_]
AttributeError: 'NoneType' object has no attribute 'seeds'

Code

from pony import orm
db = orm.Database('sqlite', ':memory:')


class Location(db.Entity):
  name = orm.Optional(str)
  parent = orm.Optional('Location', reverse='children')
  children = orm.Set('Location', cascade_delete=True)
  orm.composite_index(name, parent)

  @classmethod
  def get_root(cls) -> 'Location':
    root = cls.get(name='', parent=None)
    if not root:
      root = cls(name='', parent=None)
    return root

  @classmethod
  def get_by_path(cls, path):
    current = cls.get_root()
    for i in range(len(path)):
      current = cls.get(name=path[i], parent=current)
      if not current:
        raise ValueError(path)
    return current

  @classmethod
  def from_path(cls, path):
    parent = cls.get_by_path(path[:-1])
    entity = cls(name=path[-1], parent=parent)
    return entity

  def as_tuple(self):
    if not self.parent:
      return (None, self.name)
    return (self.parent.as_tuple(), self.name)


# XXX Remove this subclass and the error does not happen.
class Object(Location):
  pass


@orm.db_session
def list_location(path):
  # XXX Or don't use as_tuple() here.
  return (x.as_tuple() for x in Location.get_by_path(path).children)

@orm.db_session
def create_location(path):
  Location.from_path(path)

db.generate_mapping(create_tables=True)
with orm.db_session:
  Location.get_root()


# XXX Or use this context manager.
# with orm.db_session:
print(list(list_location([])))
create_location(['com.example'])
create_location(['com.example', 'foobar'])
print(list(list_location(['com.example'])))

Unfortunately, none of the ways above to prevent the issue are an option in my use case. :( Any idea how to fix this?

@NiklasRosenstein
Copy link
Author

NiklasRosenstein commented Nov 22, 2017

This is the method that the AttributeError is raised in (Line 2111):

pony/pony/orm/core.py

Lines 2104 to 2113 in ee73a6d

def get(attr, obj):
if attr.pk_offset is None and obj._status_ in ('deleted', 'cancelled'):
throw_object_was_deleted(obj)
vals = obj._vals_
if vals is None: throw_db_session_is_over('read value of', obj, attr)
val = vals[attr] if attr in vals else attr.load(obj)
if val is not None and attr.reverse and val._subclasses_ and val._status_ not in ('deleted', 'cancelled'):
seeds = obj._session_cache_.seeds[val._pk_attrs_]
if val in seeds: val._load_()
return val

The condition above that line explicitly checks if there's any subclasses of the val entity, so this is definitely related to entity subclasses. Looks like a bug to me. Can anyone confirm?

@NiklasRosenstein
Copy link
Author

NiklasRosenstein commented Nov 22, 2017

Adding

and obj._session_cache_:

to like 2110 makes it work. Is that an appropriate fix? I don't know the whole picture of this method and its implications.

@kozlovsky kozlovsky added the bug label Nov 22, 2017
@kozlovsky kozlovsky self-assigned this Nov 22, 2017
@kozlovsky kozlovsky added this to the 0.7.4 milestone Nov 22, 2017
@kozlovsky
Copy link
Member

kozlovsky commented Nov 22, 2017

Thanks for reporting!

Is that an appropriate fix?

Yes, this is correct fix. But I want to explain what is really happened:

When you work with Pony, all operations with objects should took place inside a db_session. During the db_session Pony maintains a cache with loaded objects. This way Pony avoids repeated loading of objects which were already loaded in current db_session. So, you need to wrap function with @db_session decorator or use db_session context manager. If several sequential actions are part of the same logical sequence, it is more efficient to wrap them in a single db_session.

The last line of your test example is

print(list(list_location(['com.example'])))

In this line you execute list_location function which is wrapped with @db_session decorator. But the function returns a generator object:

@orm.db_session
def list_location(path):
  return (x.as_tuple() for x in Location.get_by_path(path).children)

In Python, generators are evaluated lazily. That is, when the function returns a generator object, it is not fully executed yet. Specifically, x.as_tuple() expression is not evaluated yet. But when we returned from the function, db_session decorator thinks that all work with the database is already done, and starts clearing the session cache. At this moment, Pony assumes that no new operations will be performed with previously retrieved objects and clears some internal structures in order to simplify work for Python garbage collector.

After that, list function call placed outside of list_location function receives a generator and starts performing next calls on it. The generator starts executing x.as_tuple() expression and fail with error because some internal structures were already cleared.

At this moment Pony allows reading previously loaded attributes of objects even after db_session was finished, because many users still rely on that, but maybe in the future we will switch to "strict" default mode when all working with objects outside of db_session is forbidden (currently it is possible to enforce "strict" mode by specifying strict=True option to db_session). The benefits of strict mode is more aggressive garbage collection, slightly faster performance and lower memory usage.

Returning to your code: I think it is better to replace generator with list comprehension. List comprehensions are evaluated eagerly, and this way we can guarantee that all evaluation will take place inside db_session:

@orm.db_session
def list_location(path):
  return [x.as_tuple() for x in Location.get_by_path(path).children]

But after the fix I commited, previous version of code should work too

kozlovsky added a commit that referenced this issue Jul 23, 2018
# Major features

* Hybrid methods and properties added: https://docs.ponyorm.com/entities.html#hybrid-methods-and-properties
* Allow to base queries on another queries: `select(x.a for x in prev_query if x.b)`
* Added support of Python 3.7
* Added support of PyPy
* `group_concat()` aggregate function added
* pony.flask subpackage added for integration with Flask

# Other features

* `distinct` option added to aggregate functions
* Support of explicit casting to `float` and `bool` in queries

# Improvements

* Apply @cut_traceback decorator only when pony.MODE is 'INTERACTIVE'

# Bugfixes

* In SQLite3 `LIKE` is case sensitive now
* #249: Fix incorrect mixin used for Timedelta
* #251: correct dealing with qualified table names
* #301: Fix aggregation over JSON Column
* #306: Support of frozenset constants added
* #308: Fixed an error when assigning JSON attribute value to the same attribute: obj.json_attr = obj.json_attr
* #313: Fix missed retry on exception raised during db_session.__exit__
* #314: Fix AttributeError: 'NoneType' object has no attribute 'seeds'
* #315: Fix attribute lifting for JSON attributes
* #321: Fix KeyError on obj.delete()
* #325: duplicating percentage sign in raw SQL queries without parameters
* #331: Overriding __len__ in entity fails
* #336: entity declaration serialization
* #357: reconnect after PostgreSQL server closed the connection unexpectedly
* Fix Python implementation of between() function and rename arguments: between(a, x, y) -> between(x, a, b)
* Fix retry handling: in PostgreSQL and Oracle an error can be raised during commit
* Fix optimistic update checks for composite foreign keys
* Don't raise OptimisticCheckError if db_session is not optimistic
* Handling incorrect datetime values in MySQL
* Improved ImportError exception messages when MySQLdb, pymysql, psycopg2 or psycopg2cffi driver was not found
* desc() function fixed to allow reverse its effect by calling desc(desc(x))
* __contains__ method should check if objects belong to the same db_session
* Fix pony.MODE detection; mod_wsgi detection according to official doc
* A lot of inner fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants