Exception 'Exists' object has no attribute 'rhs'

[shield007]

What happened?

We are running django-cachalot in production and keep getting exceptions "'Exists' object has no attribute 'rhs'". This is captured in sentry, so we can see some of the details.

It's comming from the following line in cachalot/utils.py:
rhs = child.rhs

Here are some of the details set at this point:

• child - django.db.models.expressions.Exists object
• child_class - django.db.models.expressions.Exists
• children - [django.db.models.lookups.Exact,django.db.models.expressions.Exists]
• rhs - uuid.UUID
• rhs_class - uuid.UUID

What should've happened instead?

No exception

Steps to reproduce

Seems to happen on DB quieries that make use of django.db.models.expressions.Exists.

Django==3.0.6
Postgres DB (12.x)

[Andrew-Chen-Wang]

Interesting. Are you on Cachalot 2.2.0, the latest? It seemed like a problem post Django 2.2 and it should’ve been fixed... Will look into it if it’s not the version issue.

…

On Tue, Jun 23, 2020 at 4:18 AM John-Paul Stanford ***@***.***> wrote: What happened? We are running django-cachalot in production and keep getting exceptions "'Exists' object has no attribute 'rhs'". This is captured in sentry, so we can see some of the details. It's comming from the following line in cachalot/utils.py: rhs = child.rhs Here are some of the details set at this point: - child - django.db.models.expressions.Exists object - child_class - django.db.models.expressions.Exists - children - [django.db.models.lookups.Exact,django.db.models.expressions.Exists] - rhs - uuid.UUID - rhs_class - uuid.UUID What should've happened instead? No exception Steps to reproduce Seems to happen on DB quieries that make use of django.db.models.expressions.Exists. Django==3.0.6 Postgres DB (12.x) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#156>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOLG4VVOUIRRWNY7OF5J3UTRYBQNHANCNFSM4OFNIFRQ> .

[shield007]

Yes, we should be on the latest. I have the django-cachalot==2.2.0 in the requirements file.

[Andrew-Chen-Wang]

@shield007 Hm. #132 (that was merged in a different commit) was supposed to fix Django's extensions. I'll look into it. Do you mind posting traceback? It's fine if you don't; you may not need to.

[shield007]

@Andrew-Chen-Wang Sure, here is the traceback we got, I've striped out the lins from our code base. But if thats a problem let me know. It's probally fine to include them. Just taking them out while they are not needed:

AttributeError: 'Exists' object has no attribute 'rhs'
File "django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "newrelic/hooks/framework_django.py", line 540, in wrapper
return wrapped(*args, **kwargs)

removed application code

File "django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)

removed application code

File "newrelic/hooks/framework_django.py", line 931, in wrapper
return wrapped(*args, **kwargs)
File "django/views/generic/base.py", line 97, in dispatch
return handler(request, *args, **kwargs)

removed application code

File "django/db/models/query.py", line 276, in iter
self._fetch_all()
File "django/db/models/query.py", line 1261, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "django/db/models/query.py", line 57, in iter
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "cachalot/monkey_patch.py", line 36, in inner
return original(compiler, *args, **kwargs)
File "cachalot/monkey_patch.py", line 83, in inner
table_cache_keys = _get_table_cache_keys(compiler)
File "cachalot/utils.py", line 196, in _get_table_cache_keys
for t in _get_tables(db_alias, compiler.query)]
File "cachalot/utils.py", line 172, in _get_tables
for subquery in _find_subqueries_in_where(query.where.children):
File "cachalot/utils.py", line 117, in _find_subqueries_in_where
rhs = child.rhs

Also because we have this captured in sentry, I can access any of the local varibles anywhere though that backtrace stack. So let me know if you need them to help debuging.

[shield007]

Also dobule checked the running version and it is indeed 2.2.0.

/app # pip freeze | grep cachalot
django-cachalot==2.2.0

[Andrew-Chen-Wang]

@shield007 Thanks for adding your traceback. I understand the problem. Please look out for cachalot 2.2.2. The problem WAS fixed, but I apparently didn't put it in cachalot 2.2.0. It's actually fixed in master branch. Sorry for the inconvenience. That patch for 2.2.2 should come out today.

[Andrew-Chen-Wang]

@shield007 I'm going to close this issue now since I just published version 2.2.2 which should come with the fix. Sorry for the inconvenience! Please reopen this if the issue persists.

[shield007]

Thanks for looking into it. Will test 2.2.2 today.

[shield007]

We still see problems with the 2.2.2 relase. Here is a stack we are getting:

AttributeError: 'Exists' object has no attribute 'rhs'
  
  File "django/db/models/query.py", line 1261, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "django/db/models/query.py", line 57, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "cachalot/monkey_patch.py", line 29, in inner
    return original(compiler, *args, **kwargs)
  File "cachalot/monkey_patch.py", line 76, in inner
    table_cache_keys = _get_table_cache_keys(compiler)
  File "cachalot/utils.py", line 191, in _get_table_cache_keys
    for t in _get_tables(db_alias, compiler.query)]
  File "cachalot/utils.py", line 167, in _get_tables
    for subquery in _find_subqueries_in_where(query.where.children):
  File "cachalot/utils.py", line 109, in _find_subqueries_in_where
    rhs = child.rhs

[shield007]

Can this issue be reopened ?

[Andrew-Chen-Wang]

I see. Same problem. Perhaps, if we moved the subquery into the try block? @shield007 Do you mind posting a sample query? I'd like to get all subqueries fixed because it seems to be a recurring issue since Django 2.2.

[shield007]

I've looked into this a bit, I know have some code to reproduce the problem from the django shell. From what I can see the main issue is that on line 117, rhs = child.rhs. The problem is that the child does not have a property rhs because child.class is actually django.db.models.expressions.Exists.

[shield007]

I've now created a pull request, #157. This fixes it for me. Hopefully this helps with the issue.

[Andrew-Chen-Wang]

@shield007 Yes, but I didn't want #157 to invalidate Exists subqueries from working with cachalot. The deleted lines in your PR was there so that your subquery could work with cachalot in Django 2.2. I'd much rather find a solution to getting it to work than for it to not work. You can use #155 whenever someone gets around to working on disabling cachalot temporarily via a context manager. Hopefully soon from my end.

[Andrew-Chen-Wang]

@shield007 I've merged your PR for version 2.3.0. If it's still not working, you can use the new context manager to disable cachalot. Please let me know if there are any problems.

[Andrew-Chen-Wang]

ping