Improve relation choices limiting

[rubendura]

Building upon #3313 comments. This is a follow up for #3309.

There can be some memory issues with RelatedField.choices when dealing with really big querysets (~400k items in my case). This property fetches all models in the queryset and then builds a massive OrderedDict with them, which basically can clog the memory of many systems.

I suggest that we slice the queryset using html_cutoff before evaluating the queryset and building the returned OrderedDict. It seems to do the trick for me, but it'd be better if someone can confirm if this is a valid solution.

This is the code snippet that I used to do a quick (and so far successful) test. I've only added the bit slicing the queryset.

In relations.py:144 (RelatedField.choices)

@property
def choices(self):
    queryset = self.get_queryset()
    if queryset is None:
        # Ensure that field.choices returns something sensible
        # even when accessed with a read-only field.
        return {}

    return OrderedDict([
        (
            six.text_type(self.to_representation(item)),
            self.display_value(item)
        )
        for item in queryset[:self.html_cutoff]
    ])

[blag]

According to the docs, html_cutoff can be None, so I think the slicing needs to be done in a conditional. That's really all my PR #3877 does differently.

[tomchristie]

Milestoning each of this, #3877 and #3330 for review.

[wimglenn]

@blag slicing like [:None] is OK though, isn't it?

[blag]

@wimglenn: I didn't know this, but apparently yes it is. Thanks for pointing this out! 😄

I'll try to update my PR.