Failed migration from 'BaseQuerySet.aggregate_sum()' to 'BaseQuerySet.sum()'

[zeezdev]

I use models based on QuerySet class in my web application. After update to from 0.10.6 to 0.10.7 I started getting errors: AttributeError: 'MyQuerySet' object has no attribute 'aggregate_sum' because 'aggregate_sum' has been removed in new version from BaseQuerySet class.

Why 'aggregate_sum' method has been removed without prior mark as deprecated?

What should I do? Use 'BaseQuerySet.sum' method based on map-reduce or realize their own method of replacing it?

[lafrech]

See changelog:

Replaced map-reduce based QuerySet.sum/average with aggregation-based implementations #1336

Useless old map-reduce methods have been removed and aggregation-based methods have been renamed as simply sum/average.

You're right it might have been wise to keep the old names as deprecated aliases.

Just rename your methods.

[zeezdev]

Strangely enough, but in my case, a simple change of name of the method did not work.
My model field:

total = IntField(db_field='t', default=0)

my using aggregate_sum:

total_views = self.aggregate_sum("t")

i simply rename 'aggregate_sum' -> 'sum' and caught error:

  File "/webapp/.venv/local/lib/python2.7/site-packages/mongoengine/queryset/base.py", line 1283, in sum
    field_instances = self._document._lookup_field(field_parts)
  File "/webapp/.venv/local/lib/python2.7/site-packages/mongoengine/base/document.py", line 969, in _lookup_field
    % field_name)
LookUpError: Cannot resolve field "t"

what's wrong?

[lafrech]

Surprising. I'm not sure it is related. The error seems to lie in the fact that _lookup_field fails while looking for "t", which should be there AFAIU.

You may want to show more of your code, and maybe try to call _lookup_field directly, see how it goes. And try on a field with no db_field attribute, to see if it makes a difference.

Reopening the issue as I'm not sure you can reopen yourself but you could rename it or even open a new one if you confirm the issue is unrelated to aggregate_sum -> sum.

[zeezdev]

When I call "sum" with field name of my model class ("total") not db field name ("t") I always get result == 0.

In old realization aggregate method work with db_field names, passed from 'field' argument:

    def aggregate_sum(self, field):
        """Sum over the values of the specified field.

        :param field: the field to sum over; use dot-notation to refer to
            embedded document fields

        This method is more performant than the regular `sum`, because it uses
        the aggregation framework instead of map-reduce.
        """
        result = self._document._get_collection().aggregate([
            {'$match': self._query},
            {'$group': {'_id': 'sum', 'total': {'$sum': '$' + field}}}
        ])
        if IS_PYMONGO_3:
            result = list(result)
        else:
            result = result.get('result')
        if result:
            return result[0]['total']
        return 0

In new sum() realization aggregate() works with model fields names, that resolved in 'cls._fields' without considering that field_name and db_field may be different:

https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/base/document.py#L951

I think it is bug. It is don't work if your model's fields have db_field different from model field names.

[wojcikstefan]

@zeezdev sorry about that. I take the blame for not being more cautious and starting with a deprecation warning.

As for the issue you're experiencing, it seems that the new BaseQuerySet.sum doesn't respect the db_field:

In [11]: class TestDoc(Document):
    total = IntField(db_field='t', default=0)
   ....:

In [12]: TestDoc.objects.create(total=20)
Out[12]: <TestDoc: TestDoc object>

In [13]: TestDoc.objects.create(total=10)
Out[13]: <TestDoc: TestDoc object>

In [14]: TestDoc.objects.create(total=5)
Out[14]: <TestDoc: TestDoc object>

In [15]: TestDoc.objects.sum('total')
Out[15]: 0

In [16]: TestDoc.objects.sum('t')
---------------------------------------------------------------------------
LookUpError                               Traceback (most recent call last)
<ipython-input-16-00bc26fc7158> in <module>()
----> 1 TestDoc.objects.sum('t')

/Users/wojcikstefan/Repos/temp/mongoengine/mongoengine/queryset/base.pyc in sum(self, field)
   1281         ListField = _import_class('ListField')
   1282         field_parts = field.split('.')
-> 1283         field_instances = self._document._lookup_field(field_parts)
   1284         if isinstance(field_instances[-1], ListField):
   1285             pipeline.insert(1, {'$unwind': '$' + field})

/Users/wojcikstefan/Repos/temp/mongoengine/mongoengine/base/document.pyc in _lookup_field(cls, parts)
    967                 else:
    968                     raise LookUpError('Cannot resolve field "%s"'
--> 969                                       % field_name)
    970             else:
    971                 ReferenceField = _import_class('ReferenceField')

LookUpError: Cannot resolve field "t"

Working on a fix as we speak!