The Datalookup library makes it easier to filter and manipulate your data. The module is inspired by the Django Queryset Api and it's lookups.
$ pip install datalookupThroughout the below examples, we'll refer to the following data, which comprise a list of authors with the books they wrote.
data = [
{
"id": 1,
"author": "J. K. Rowling",
"books": [
{
"name": "Harry Potter and the Chamber of Secrets",
"genre": "Fantasy",
"published": "1998"
},
{
"name": "Harry Potter and the Prisoner of Azkaban",
"genre": "Fantasy",
"published": "1999"
}
]
},
{
"id": 2,
"author": "Agatha Christie",
"books": [
{
"name": "And Then There Were None",
"genre": "Mystery",
"published": "1939"
}
]
}
]Datalookup makes it easy to find an author by calling one of the methods of the Dataset class like filter() or exclude(). There are multiple ways to retrieve an author.
Use one of the field of your author dictionary to filter your data.
from datalookup import Dataset
# Use Dataset to manipulate and filter your data
books = Dataset(data)
# Retrieve an author using the author name
authors = books.filter(author="J. K. Rowling")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
# Retrieve an author using '__in' lookup
authors = books.filter(id__in=[2, 3])
assert len(authors) == 1
assert authors[0].author == "Agatha Christie"
# Retrieve an author using 'exclude' and '__contains' lookup
authors = books.exclude(author__contains="Christie")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"Use a related field like books separated by a __ (double-underscore) and a field of the books. Something like books__name.
# Retrieve an author using the date when the book was published
authors = books.filter(books__published="1939")
assert len(authors) == 1
assert authors[0].author == "Agatha Christie"
# Retrieve an author using '__regex' lookup
authors = books.filter(books__name__regex=".*Potter.*")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"Keyword argument queries - in filter(), etc. - are "AND"ed together. If you need to execute more complex queries (for example, queries with OR statements), you can combine two filter request with "|".
# Retrieve an author using multiple filters with a single request (AND). This
# filter use the '__icontains' lookup. Same as '__contains' but case-insensitive
authors = books.filter(books__name__icontains="and", books__genre="Fantasy")
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
# Retrieve an author by combining filters (OR)
authors = books.filter(author="Stephane Capponi") | books.filter(
author="J. K. Rowling"
)
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"The library provides also a way to filter nested relationship. This means that you can make requests to only retrieve books in the author collection. Or you can use that output to filter the authors.
# filter_related is the method to use to filter all related nodes
related_books = books.filter_related('books', genre="Mystery")
assert len(related_books) == 1
assert related_books[0].name == "And Then There Were None"
# You can also use filter_related to filter authors.
authors = books.filter(
books=books.filter_related('books', name__regex=".*Potter.*")
)
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"Sometimes you will want to filter the author but also the related books. It is possible to do that by calling the on_cascade() method before filtering.
# Filter the author but also the books of the author
authors = books.on_cascade().filter(
books__name="Harry Potter and the Chamber of Secrets"
)
assert len(authors) == 1
assert authors[0].author == "J. K. Rowling"
# The books are also filtered
assert len(authors[0].books) == 1
assert authors[0].books[0].name == "Harry Potter and the Chamber of Secrets"Field lookups are used to specify how a the dataset should query the results it returns. They're specified as keyword arguments to the Dataset methods filter() and exclude(). Basic lookups keyword arguments take the form "field__lookuptype=value". (That's a double-underscore).
As a convenience when no lookup type is provided (like in books.filter(id=1)) the lookup type is assumed to be exact.
# author is one of the field of the dictionary
# '__contains' is the lookup
books.filter(author__contains="Row")| Lookup | Case-insensitive lookup | Description |
|---|---|---|
| exact | iexact | Exact match |
| contains | icontains | Containment test |
| startswtih | istartswith | Starts with a specific string |
| endswith | iendswith | Ends with a specific string |
| regex | iregex | Regular expression match |
| in | In a given iterable; strings (being iterables) are accepted | |
| gt | Grater than | |
| gte | Greater that or equal | |
| lt | Lower than | |
| lte | Lower than or equal to | |
| range | Range between two values. Integer only | |
| isnull | Check that a field is null. Takes either True or False | |
| contained_by | Check data is a subset of the passed values. ArrayField only | |
| overlap | Data shares any results with the passed values. ArrayField only | |
| len | Check length of the array. ArrayField only |
Datalookup does not stop here. The full documentation is in the docs directory or online at https://datalookup.readthedocs.io/en/latest/
Anyone can contribute to Datalookup's development. Checkout our documentation on how to get involved: https://datalookup.readthedocs.io/en/latest/internals/contributing.html
Copyright Stephane Capponi and others, 2023 Distributed under the terms of the MIT license, Datalookup is free and open source software.
Datalookup was inspired by Django and only the RegisterLookupMixin was copied. Everything else was inspired and re-interpreted. You can find the license of Django in the licenses folder.