Object not found.

[luisdemarchi]

There is some moment that it is not possible to return an object and everything spoils. When I try to pull a list with all objects nothing else works. I could not detect exactly when this occurs, sometimes it looks like it was after using delete(). But I can say that it is unstable.

# Presence REDIS DB (no-SQL)
class Presence(walrus.Model):
    __database__ = redis_db
    channel_name = walrus.TextField(primary_key=True)
    user = walrus.UUIDField(index=True, default='00000000-0000-0000-0000-000000000000')
    last_seen = walrus.DateTimeField(default=datetime.datetime.now, index=True)
    is_accessible = walrus.IntegerField(index=True, default=0)
    connection_state = walrus.IntegerField(default=1, index=True)

>>> for t in Presence.all():
        print(t.user, t.is_accessible, t.connection_state)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 789, in all
    yield cls.load(result, convert_key=False)
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
    raise KeyError('Object not found.')
KeyError: 'Object not found.'

---

See that in the object list I have an object that is marked as indexable, when I do a query looking specifically for that object it also bursts the same error.

>>> for presence in Presence.all():
        print("pint user ID: {}".format(presence.user))
pint user ID: 46981222-d147-48de-aba2-e63133d2491d

>>> for t in Presence.query(Presence.user == '46981222-d147-48de-aba2-e63133d2491d'):
       print("pint user ID: {}".format(presence.user))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 839, in query
    yield cls.load(hash_id, convert_key=False)
  File "/Volumes/MacHD/Users/Git/projectX-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
    raise KeyError('Object not found.')
KeyError: 'Object not found.'

or: uuid.UUID('46981222-d147-48de-aba2-e63133d2491d')):

---

After the error starts happening, not even the method to delete all records of that type of object does:

>>> Presence.query_delete()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/Volumes/MacHD/Users/Git/webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 857, in query_delete
    cls.load(hash_id, convert_key=False).delete()
  File "/Volumes/MacHD/Users/Git/-webservice/.env/lib/python3.6/site-packages/walrus/models.py", line 889, in load
    raise KeyError('Object not found.')
KeyError: 'Object not found.'

[jerinzam]

Any solution for this

[coleifer]

That's really odd... can you set up a script I can try to reproduce this?

[johndlong]

Just speculating, but are there any save operations going on in a separate thread/process?

The model save operation does a delete as part of the update process. This could potentially introduce very small timing windows where other processes would experience exactly these failures.

[johndlong]

Another thought. Would a lua script that sets all the key/value pairs of a model in an atomic operation potentially avoid having to do that delete operation during a save?

[coleifer]

👍

[rudaporto]

I have the same issue when using a Model to share status between two or more workers.

So, can I say that now Walrus models are not thread-safe?

[coleifer]

So, can I say that now Walrus models are not thread-safe?

Has nothing to do with thread-safety. Redis is single-threaded, after all.

[lovetoburnswhen]

I also run into KeyErrors when there are two workers interacting with the same db: one is iterating through a Model.query() result while the other one is modifying/deleting model objects.

[coleifer]

Yea there's no isolation. I strongly advise to use a relational db.

[jhorman]

I am experiencing this as well. Isn't this something solved by doing the save using a pipeline. I think the problem is just finding the data in one index before it is in another. Pipelines atomically write.

[lovetoburnswhen]

I got around this by

1. Managing my own "namespaces" with a field on the Model to enforce isolation by restricting each client to a certain namespace.

2. Bypassing the Model API and interacting with Redis directly (only appropriate for certain cases). Huge performance boost since instead of doing model.load(...) for model in models... you can do something like:

with db.pipeline() as pipe:
    for id_key in id_keys:
        pipe.hgetall(id_key)

    model_data_dicts = pipe.execute()

[jhorman]

Yeah, definitely a nice performance boost, but sort of sep from the main issue. As is, without using a basic pipeline, I think save is basically broken. Iterating the collection from any other thread/process can fail since it isn't fully written to redis yet.

[coleifer]

I strongly advise to use a relational db.

No one should be using the walrus models code. I need to put a warning in the docs to that effect. It was an experiment and yeah it kinda works, but it's definitely nothing I would ever use myself and that is a big red flag to me.

[jhorman]

That statement confuses me honestly. There are many use cases for Redis in which a modeled approach to transient data makes sense, to hide the complexity of the various data structures. That is how I landed here. I was doing the sorts of things you are doing, on my own, but then found Walrus. I have a case in which I most definitely do not want a relational database (very high throughput of transient objects).

I get though that it is your project and you want to deprecate that feature.

[coleifer]

55dc768

https://walrus.readthedocs.io/en/latest/models.html

deprecate that feature.

I do not intend to "deprecate" anything, and I did not intend to give that impression. What I do want to convey is that the implementation has a lot of rough edges and is too flaky for me to ever want to use it. I feel it is necessary to put a warning up that the code is "experimental"-quality, so that people can re-evaluate walrus if they are expecting some kind of stability guarantee.