Elasticsearch filterset

[alejandronunez]

The objective of this pull request is to add support for ElasticSearch, specifically, to generate fields for GraphQL queries based on a description over indexes of ElasticSearch, which can be used to filter the ElasticSearch data.

Problem

django-filter is a useful extension for Django that generates inputs of form based on the description of the Django models, afterward with a Django queryset it is able to generate the necessary queries to resolve the filtered data. graphene-django has an extension that uses that functionality changing the generation of fields of Graphene instead of the inputs of form. Currently, there is no equivalent of this extension with support for Elasticsearch. The work on this pull request adds that support and allows for the generation of fields for Graphene, based on the index of Elasticsearch it generates the necessary queries for Elasticsearch.

Proposal Solution

Using django-elasticsearch-dsl, we have the foundation needed to generate the indexes based on the Django Models and a manager to do query ElasticSearch using elasticsearch-dsl queries. An option would be to implement an adapter for a manager of ElasticSearch that behaves as a Django queryset, of course, that would be hard, it is out of the scope of that library and we would not have the capability of doing queries using advance features of ElasticSearch.

Therefore, I decided to extend graphene-django which generates filters, and use new lookups to generate the fields of Graphene in order to build the ElasticSearch queries. This will allow us to use specific features of ElasticSearch, such as fussy queries and regex queries.

Specific Solution

We have a proxy component to replace the use of the Django model manager for an ElasticSearch manager and the Django queryset for a ElasticSearch search. A new DjangoESFilterConnectionField that extends from DjangoFilterConnectionField will be responsible for the replacement. django-elasticsearch-dsl returns a Django queryset as result of a query to ElasticSearch, so that when QuerysetProxy is asked to be evaluated, firstly it does the query to ElasticSearch and returns the resultant queryset.

A new FilterSetES could be used as base class by new filters to describe which ElasticSearch document must be used, which fields of that document must be generated and also how those fields will behave. Besides, this filterset has all the logic needed to do that description as declared fields in the class or using injection on the metadata of the class.

The FilterES instances have all the information to describe a field for FilterSetES and delegate the generation of fields and queries to several processors, each type of FilterES has a default processor and you can specify as many as you need. For example, it would be interesting that an username filter field for the model User can be sought using exact match, fussy search and regular expressions. Each processor generates a Graphene field.

FilterES makes a chain of processors based on the specification of the lookups (processor) and using a method as a decorator, it generates the Graphene fields with their respective types. Furthermore, with an observable object, this FilterES attaches to field their respective processor to generate the ElasticSearch query from the arguments passed to those Graphene fields.

Some design patterns used:

• Proxy: To do the ES query before returning the queryset.
• Simple Factory: To make new Processor based on name
• Decorator: To have a processor chain where each one adds its field to the list fields in the introspection.
• Observable: To have the fields attach with their resolvers and use them for generating the ES query.

[jkimbo]

Wow this looks great @alejandronunez ! I’m not very familiar with elasticsearch so it might take a while for this to be reviewed.

I had one question before that though: would this be better as a stand alone library? My concern is that if it gets merged into graphene-django then it will have to be maintained alone with the rest of the library. At first glance the code looks pretty self contained and if it was its own library then it could directly require the Django-elasticsearch-dsl library rather than having to detect it. It could also live in the graphql-python organisation and you could be setup as admin on the repo. What do you think?

[jkimbo]

@alejandronunez what do you think about releasing this as a separate library?

[BossGrand]

@alejandronunez what do you think about releasing this as a separate library?

I agree with this sentiment seems a bit heavy to throw straight into graphene-django

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.