Filter object_list later in serialize_to_json for easy calculation of whole dataset

[foobar76]

First tnx for this great django binding! I integrated it and wanted something similar as her: http://datatables.net/examples/advanced_init/footer_callback.html. As soon I edited it for my needs I realized, that the view (of course) paginates the data sending to the table, so that no calculation on the WHOLE dataset is possible in the client in js.
So I googled again and came up, with the possibility of giving custom server side data to the table: http://stackoverflow.com/questions/6090260/jquery-datatables-return-additional-information-from-server (should be faster anyway)

So I came up with the following solution (sorry for not making a diff - first post her on github):

removing the last lines in apply_queryset_options and moving them to

def serialize_to_json(self, object_list):

      [...]

        object_list, total_records, unpaged_total = object_list

        options = self._get_datatable_options()

        # Do some things here with your FULL object list
        debit_rental = 0
        credit_rental = 0
        sum_rental = 0

        for obj in object_list:
            if obj.debit:
                debit_rental += obj.rental
            else:
                credit_rental += obj.rental
        sum_rental = credit_rental - debit_rental

        full_response = {'sum_rental' : '{0:.2f}'.format(sum_rental),
                        }
       # Filter here not in apply_queryset_options
        if options.page_length != -1:
            i_begin = options.start_offset
            i_end = options.start_offset + options.page_length
            object_list = object_list[i_begin:i_end]

        # Do something with filtered object_list if needed

        response_obj = {
            'sEcho': self.request.GET.get('sEcho', None),
            'iTotalRecords': total_records,
            'iTotalDisplayRecords': unpaged_total,
            'aaData': [self.get_record_data(obj) for obj in object_list],
        }

        #Add Dict to response
        response_obj.update(full_response)

        return json.dumps(response_obj, indent=4)

With this edit it would be easy to do so with just overwriting serialize_to_json.

[tiliv]

Hello! Thanks for describing what you were hoping to do.

Commits 0ff5d41 and 8abeb12 should address this issue.

I've added a method called get_json_response_object() which builds the dictionary that will get serialized. It receives the filtered object_list and then handles pagination, and finally returns the python dictionary that will later be serialized.

This allows you to call super() to get the normal response dictionary, and then just add to it. In your case, you can examine object_list and it will not be limited by the paging. (If you still wanted to look at the paging, you can run it through self.paginate_object_list(object_list) again, but be wary of performance issues, since object_list can potentially be a queryset, not a list, for query performance boosts.)

This should now be possible if you pull from the latest dev commits, copied from your example:

class MyDatatableView(DatatableView):
    datatable_options = { ... }

    def get_json_response_object(self, object_list, *args, **kwargs):
        data = super(MyDatatableView, self).get_json_response_object(object_list, *args, **kwargs)

        # Do some things here with your FULL object list
        debit_rental = 0
        credit_rental = 0
        sum_rental = 0

        for obj in object_list:
            if obj.debit:
                debit_rental += obj.rental
            else:
                credit_rental += obj.rental
        data['sum_rental'] = credit_rental - debit_rental

        return data

If this looks good and feels like it does the right thing, I'll close this and put it into a proper release so that you can upgrade to the latest via pip.

[foobar76]

Whow - that was fast. Tnx for making my first "report" here on github such a success story :) Exactly what I (and maybe some more users) needs to solve this case.

[foobar76]

Short addition: Since the filtering is done now inside get_json_response_object there is no chance to use super if we need the filtered object_list, but this should be no problem since get_json_response_object() is a lot shorter then overwriting the big apply_queryset_options()

[tiliv]

To be clear, the object_list parameter is indeed a filtered version (after the stuff from apply_queryset_options happens), it just hasn't been cut down to the current page of results yet. That happens inside of the call to super().get_json_response_object().

My reasoning was that it let's the paging happen as late as possible, provides access to the filtered version of object_list without paging, and is a much nicer method override in the worst case scenario.

Best of luck!