Chain multiple (disparate) QuerySets in Django
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
queryset_sequence
tests
.coveragerc
.gitignore
.travis.yml
CHANGELOG.rst
LICENSE
MANIFEST.in
README.rst
manage.py
requirements.txt
setup.cfg
setup.py
tox.ini

README.rst

Django QuerySetSequence

https://travis-ci.org/percipient/django-querysetsequence.svg?branch=master https://coveralls.io/repos/github/percipient/django-querysetsequence/badge.svg?branch=master

The QuerySetSequence wrapper helps to deal with disparate QuerySet classes, while treating them as a single QuerySet.

Supported Features

Listed below are features of Django's QuerySets that QuerySetSequence implements. The behavior should match that of QuerySet, but applied across multiple QuerySets:

  • Methods that take a list of fields (e.g. filter(), exclude(), get(), order_by()) must use fields that are common across all sub-QuerySets.
  • Relationships across related models work (e.g. 'foo__bar', 'foo', or 'foo_id'). syntax).
  • The sub-QuerySets are evaluated as late as possible (e.g. during iteration, slicing, pickling, repr()/len()/list()/bool() calls).
  • Public QuerySet API methods that are untested/unimplemented raise NotImplementedError. AttributeError is raised on attributes not explictly inherited from QuerySet.
QuerySet API implemented by QuerySetSequence
Method Implemented? Notes
filter() See [1] for information on the QuerySet lookup: '#'.
exclude() See [1] for information on the QuerySet lookup: '#'.
annotate()  
order_by() Does not support random ordering (e.g. order_by('?')). See [1] for information on the QuerySet lookup: '#'.
reverse()  
distinct()  
values()  
values_list()  
dates()  
datetimes()  
none()  
all()  
union()  
intersection()  
difference()  
select_related()  
prefetch_related()  
extra()  
defer()  
only()  
using()  
select_for_update()  
raw()  
get() See [1] for information on the QuerySet lookup: '#'.
create() Cannot be implemented in QuerySetSequence.
get_or_create() Cannot be implemented in QuerySetSequence.
update_or_create() Cannot be implemented in QuerySetSequence.
bulk_create() Cannot be implemented in QuerySetSequence.
count()  
in_bulk() Cannot be implemented in QuerySetSequence.
iterator()  
latest()  
earliest()  
first()  
last()  
aggregate()  
exists()  
update() Cannot be implemented in QuerySetSequence.
delete()  
as_manager()  
[1](1, 2, 3, 4)

QuerySetSequence supports a special field lookup that looks up the index of the QuerySet, this is represented by '#'. This can be used in any of the operations that normally take field lookups (i.e. filter(), exclude(), and get()), as well as order_by().

A few examples are below:

# Order first by QuerySet, then by the value of the 'title' field.
QuerySetSequence(...).order_by('#', 'title')

# Filter out the first QuerySet.
QuerySetSequence(...).filter(**{'#__gt': 0})

Note

Ordering first by QuerySet allows for a more optimized code path when iterating over the entries.

Warning

Not all lookups are supported when using '#' (some lookups simply don't make sense; others are just not supported). The following are allowed:

  • exact
  • iexact
  • contains
  • icontains
  • in
  • gt
  • gte
  • lt
  • lte
  • startswith
  • istartswith
  • endswith
  • iendswith
  • range

Requirements

Installation

Install the package using pip.

pip install --upgrade django-querysetsequence

Usage

# Import QuerySetSequence
from queryset_sequence import QuerySetSequence

# Create QuerySets you want to chain.
from .models import SomeModel, OtherModel

# Chain them together.
query = QuerySetSequence(SomeModel.objects.all(), OtherModel.objects.all())

# Use query as if it were a QuerySet! E.g. in a ListView.

You can also provide a model keyword argument if you need to specify the QuerySet Model, e.g. for compatibility with some third-party applications that check the model field for equality

Example

class Author(models.Model):
    name = models.CharField(max_length=50)

    class Meta:
        ordering = ['name']

    def __str__(self):
        return self.name


class Article(models.Model):
    title = models.CharField(max_length=100)
    author = models.ForeignKey(Author)

    def __str__(self):
        return "%s by %s" % (self.title, self.author)


class Book(models.Model):
    title = models.CharField(max_length=50)
    author = models.ForeignKey(Author)
    release = models.DateField(auto_now_add=True)

    def __str__(self):
        return "%s by %s" % (self.title, self.author)

# Create some data.
alice = Author.objects.create(name='Alice')
article = Article.objects.create(title='Dancing with Django', author=alice)

bob = Author.objects.create(name='Bob')
article = Article.objects.create(title='Django-isms', author=bob)
article = Book.objects.create(title='Biography', author=bob)

# Create some QuerySets.
books = Book.objects.all()
articles = Article.objects.all()

# Combine them into a single iterable.
published_works = QuerySetSequence(books, articles)

# Find Bob's titles.
bob_works = published_works.filter(author=bob)
# Still an iterable.
print([w.title for w in bob_works])  # prints: ['Biography', 'Django-isms']

# Alphabetize the QuerySet.
published_works = published_works.order_by('title')
print([w.title for w in published_works])  # prints ['Biography', 'Dancing with Django', 'Django-isms']

Django REST Framework integration

django-querysetsequence comes with a custom CursorPagination class that helps integration with Django REST Framework. It is optimized to iterate over a QuerySetSequence first by QuerySet and then by the normal ordering configuration. This uses the optimized code-path for iteration that avoids interleaving the individual QuerySets. For example:

from queryset_sequence.pagination import SequenceCursorPagination

class PublicationPagination(SequenceCursorPagination):
    ordering = ['author', 'title']

class PublicationViewSet(viewsets.ModelViewSet):
    pagination_class = PublicationPagination

    def get_queryset(self):
        # This will return all Books first, then all Articles. Each of those
        # is individually ordered by ``author``, then ``title``.
        return QuerySetSequence(Book.objects.all(), Article.objects.all())

Attribution

This is based on a few DjangoSnippets that had been going around:

Contribute

  • Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.
  • Fork the repository on GitHub to start making your changes.
  • Write a test which shows that the bug was fixed or that the feature works as expected.
  • Send a pull request and bug the maintainer until it gets merged and published.