collection_meta task is broken

[diox]

The collection_meta task that updates addon/themes counts on a Collection as well as the top tags for the collection appears to have been broken for a while. We've got hundreds of thousands of sentry tracebacks like this one:

OperationalError: (1213, 'Deadlock found when trying to get lock; try restarting transaction')
  File "celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "celery/app/trace.py", line 438, in __protected_call__
    return self.run(*args, **kwargs)
  File "olympia/bandwagon/tasks.py", line 86, in collection_meta
    all_personas=all_personas)
  File "django/db/models/query.py", line 563, in update
    rows = query.get_compiler(self.db).execute_sql(CURSOR)
  File "django/db/models/sql/compiler.py", line 1062, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "raven/contrib/django/client.py", line 114, in execute
    return real_execute(self, sql, params)
  File "django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "MySQLdb/cursors.py", line 250, in execute
    self.errorhandler(self, exc, value)
  File "MySQLdb/connections.py", line 50, in defaulterrorhandler
    raise errorvalue

[eviljeff]

Not sure I really understand why this task is written in such a strange way:
Collection.objects.filter(id=c.id).update(addon_count=addon_count, all_personas=all_personas)
rather than c.update(...)

And c.top_tags = doesn't update at all. (Not that we should care as tags are going away)

[diox]

My guess is that it's actively trying to avoid hitting the regular post_save signals... that would re-trigger the task.

[diox]

Also top_tags is an object with a __set__ so it actually does update it directly.

[ValentinaPC]

Verified as fixed on AMO-dev FF56(Win 7).
Add-ons inside a collection are counted correctly while using the new AMO frontend.
Postfix screenshot:

[EnTeQuAk]

https://sentry.prod.mozaws.net/operations/olympia-prod/issues/389934/

[diox]

Still happening in prod : https://sentry.prod.mozaws.net/operations/olympia-prod/issues/668574/

[diox]

No traceback about this for a while. @EnTeQuAk solved deadlocks in submission in #6365 a couple months ago or so, maybe it helped those deadlocks by accident ?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If you think this bug should stay open, please comment on the issue with further details. Thank you for your contributions.