Skip to content
Library to convert those MongoDB queries to something else, like a python expresion, a function or a django query (Q) object tree
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
ci
docs
src/mongoql_conv
tests
.bumpversion.cfg
.cookiecutterrc
.coveragerc
.editorconfig
.gitignore
.travis.yml
AUTHORS.rst
CHANGELOG.rst
CONTRIBUTING.rst
LICENSE
MANIFEST.in
README.rst
appveyor.yml
conftest.py
setup.cfg
setup.py
tox.ini

README.rst

Overview

docs Documentation Status
tests
Travis-CI Build Status AppVeyor Build Status Requirements Status
Coverage Status Coverage Status
package

Library to convert those MongoDB queries to something else, like a python expresion, a function or a Django Q object tree to be used with a ORM query.

For now, only supports flat operations. No subdocuments. It might work but results are undefined/buggy. Could be fixed though ...

  • Free software: BSD license

Installation

pip install mongoql-conv

Or:

pip install mongoql-conv[django]

API

to_string

>>> from mongoql_conv import to_string

>>> to_string({"myfield": 1})
"row['myfield'] == 1"

>>> to_string({})
'True'

>>> set(to_string({"field1": 1, "field2": 2}).split(' and ')) == {"(row['field2'] == 2)", "(row['field1'] == 1)"}
True

>>> to_string({"myfield": 1}, object_name='item')
"item['myfield'] == 1"

>>> to_string({"myfield": {"$in": [1, 2]}})
"row['myfield'] in {1, 2}"

>>> to_string({"myfield": {"$in": {1: 2}}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_string({"myfield": {"$and": []}})
'True'

to_string: Supported operators

to_string: Supported operators: Arithmetic

  • $gt:

    >>> to_string({"myfield": {"$gt": 1}})
    "row['myfield'] > 1"
    >>> to_string({"myfield": {"$gt": [1]}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part [1]. Expected one of: int, ...float, str, ...bool, None.
    
  • $gte:

    >>> to_string({"myfield": {"$gte": 1}})
    "row['myfield'] >= 1"
    
  • $lt:

    >>> to_string({"myfield": {"$lt": 1}})
    "row['myfield'] < 1"
    
  • $lte:

    >>> to_string({"myfield": {"$lte": 1}})
    "row['myfield'] <= 1"
    
  • $eq:

    >>> to_string({"myfield": {"$eq": 1}})
    "row['myfield'] == 1"
    >>> to_string({"myfield": 1})
    "row['myfield'] == 1"
    
  • $ne:

    >>> to_string({"myfield": {"$ne": 1}})
    "row['myfield'] != 1"
    
  • $mod:

    >>> to_string({"myfield": {"$mod": [2, 1]}})
    "row['myfield'] % 2 == 1"
    >>> to_string({"myfield": {"$mod": [2, 1, 3]}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
    >>> to_string({"myfield": {"$mod": 2}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
    >>> to_string({"myfield": {"$mod": (2, 1)}})
    "row['myfield'] % 2 == 1"
    

to_string: Supported operators: Containers

  • $in:

    >>> to_string({"myfield": {"$in": (1, 2, 3)}})
    "row['myfield'] in {1, 2, 3}"
    
  • $nin:

    >>> to_string({"myfield": {"$nin": [1, 2, 3]}})
    "row['myfield'] not in {1, 2, 3}"
    >>> to_string({"myfield": {"$nin": {1: 2}}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
    
  • $size:

    >>> to_string({"myfield": {"$size": 3}})
    "len(row['myfield']) == 3"
    >>> to_string({"myfield": {"$size": "3"}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part '3'. Expected one of: int...
    
  • $all:

    >>> to_string({"myfield": {"$all": [1, 2, 3]}})
    "set(row['myfield']) >= {1, 2, 3}"
    >>> to_string({"myfield": {"$all": 1}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    
  • $exists:

    >>> to_string({"myfield": {"$exists": True}})
    "'myfield' in row"
    >>> to_string({"myfield": {"$exists": False}})
    "'myfield' not in row"
    

to_string: Supported operators: Boolean operators

  • $or:

    >>> to_string({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
    "(row['bubu'] > 1) or (row['bubu'] < 2)"
    >>> to_string({'$or': "invalid value"})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
  • $and:

    >>> to_string({'$and':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]})
    "(row['bubu'] > 1) and (row['bubu'] < 2)"
    >>> to_string({'$or': "invalid value"})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
  • $*nesting*:

    >>> to_string({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]})
    "(row['bubu'] > 1) and ((row['bubu'] < 2) or ((row['bubu'] < 3) and (row['bubu'] < 4)))"
    

to_string: Supported operators: Regular expressions

  • $regex:

    >>> to_string({"myfield": {"$regex": 'a'}})
    "re.search('a', row['myfield'], 0)"
    
    >>> to_string({"bubu": {"$regex": ".*x"}}, object_name='X')
    "re.search('.*x', X['bubu'], 0)"
    
    >>> to_string({"myfield": {"$regex": 'a', "$options": 'i'}})
    "re.search('a', row['myfield'], ...2...)"
    
    >>> closure = {}
    >>> to_string({"bubu": {"$regex": ".*x"}}, closure=closure), closure
    ("var0.search(row['bubu'])", {'var0': "re.compile('.*x', 0)"})
    
    >>> to_string({"myfield": {"$regex": 'junk('}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
    
    
    >>> to_string({"myfield": {"$regex": 'a', 'junk': 'junk'}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> set(to_string({"myfield": {"$regex": 'a', '$nin': ['aaa']}}).split(' and ')) == {
    ...     "(re.search('a', row['myfield'], 0))",
    ...     "(row['myfield'] not in {'aaa'})"
    ... }
    True
    
    >>> to_string({"bubu": {"$regex": ".*", "$options": "junk"}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
    
    >>> to_string({"bubu": {"$options": "i"}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    

to_string (lax mode)

to_string (lax mode): Supported operators: Containers

  • $all:

    >>> to_string({"myfield": {"$all": [1, 2, 3]}}, lax=True)
    "set(row.get('myfield', LaxNone)) >= {1, 2, 3}"
    >>> to_string({"myfield": {"$all": 1}})
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    

to_func

>>> from mongoql_conv import to_func

>>> to_func({"myfield": 1}).source
"lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"

>>> to_func({}).source
'lambda item: (True) # compiled from {}'

>>> list(filter(to_func({"myfield": 1}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}]

>>> list(filter(to_func({}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": [1, 2]}}).source
"lambda item, var0={1, 2}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': [1, 2]}}"

>>> list(filter(to_func({"myfield": {"$in": [1, 2]}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": {1: 2}}}).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_func({"myfield": {"$and": []}}).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"

>>> list(filter(to_func({"myfield": {"$and": []}}), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

to_func: Supported operators

to_func: Supported operators: Arithmetic

  • $gt:

    >>> to_func({"myfield": {"$gt": 1}}).source
    "lambda item: (item['myfield'] > 1) # compiled from {'myfield': {'$gt': 1}}"
    >>> to_func({"myfield": {"$gt": [1]}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part [1]. Expected one of: int,...float, str,...bool, None.
    
    >>> list(filter(to_func({"myfield": {"$gt": 1}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]
    
  • $gte:

    >>> to_func({"myfield": {"$gte": 1}}).source
    "lambda item: (item['myfield'] >= 1) # compiled from {'myfield': {'$gte': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$gte": 2}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 2}, {'myfield': 3}, {'myfield': 4}]
    
  • $lt:

    >>> to_func({"myfield": {"$lt": 1}}).source
    "lambda item: (item['myfield'] < 1) # compiled from {'myfield': {'$lt': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$lt": 1}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}]
    
  • $lte:

    >>> to_func({"myfield": {"$lte": 1}}).source
    "lambda item: (item['myfield'] <= 1) # compiled from {'myfield': {'$lte': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$lte": 1}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}, {'myfield': 1}]
    
  • $eq:

    >>> to_func({"myfield": {"$eq": 1}}).source
    "lambda item: (item['myfield'] == 1) # compiled from {'myfield': {'$eq': 1}}"
    >>> to_func({"myfield": 1}).source
    "lambda item: (item['myfield'] == 1) # compiled from {'myfield': 1}"
    
    >>> list(filter(to_func({"myfield": {"$eq": 2}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 2}]
    
  • $ne:

    >>> to_func({"myfield": {"$ne": 1}}).source
    "lambda item: (item['myfield'] != 1) # compiled from {'myfield': {'$ne': 1}}"
    
    >>> list(filter(to_func({"myfield": {"$ne": 2}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}, {'myfield': 1}, {'myfield': 3}, {'myfield': 4}]
    
  • $mod:

    >>> to_func({"myfield": {"$mod": [2, 1]}}).source
    "lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
    >>> to_func({"myfield": {"$mod": [2, 1, 3]}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
    
    >>> to_func({"myfield": {"$mod": 2}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
    
    >>> to_func({"myfield": {"$mod": (2, 1)}}).source
    "lambda item: (item['myfield'] % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"
    
    >>> list(filter(to_func({"myfield": {"$mod": (2, 1)}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 1}, {'myfield': 3}]
    

to_func: Supported operators: Containers

  • $in:

    >>> to_func({"myfield": {"$in": (1, 2, 3)}}).source
    "lambda item, var0={1, 2, 3}: (item['myfield'] in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"
    
    >>> list(filter(to_func({"myfield": {"$in": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 1}, {'myfield': 2}, {'myfield': 3}]
    
  • $nin:

    >>> to_func({"myfield": {"$nin": [1, 2, 3]}}).source
    "lambda item, var0={1, 2, 3}: (item['myfield'] not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$nin": {1: 2}}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"myfield": {"$nin": (1, 2, 3)}}), [{"myfield": i} for i in range(5)]))
    [{'myfield': 0}, {'myfield': 4}]
    
  • $size:

    >>> to_func({"myfield": {"$size": 3}}).source
    "lambda item: (len(item['myfield']) == 3) # compiled from {'myfield': {'$size': 3}}"
    
    >>> to_func({"myfield": {"$size": "3"}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part '3'. Expected one of: int...
    
    >>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": 'x'*i} for i in range(5)]))
    [{'myfield': 'xxx'}]
    
    >>> list(filter(to_func({"myfield": {"$size": 3}}), [{"myfield": list(range(i))} for i in range(5)]))
    [{'myfield': [0, 1, 2]}]
    
  • $all:

    >>> to_func({"myfield": {"$all": [1, 2, 3]}}).source
    "lambda item, var0={1, 2, 3}: (set(item['myfield']) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$all": 1}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"myfield": {"$all": [3, 4]}}), [{"myfield": list(range(i))} for i in range(7)]))
    [{'myfield': [0, 1, 2, 3, 4]}, {'myfield': [0, 1, 2, 3, 4, 5]}]
    
  • $exists:

    >>> to_func({"myfield": {"$exists": True}}).source
    "lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"
    
    >>> to_func({"myfield": {"$exists": False}}).source
    "lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"
    
    >>> list(filter(to_func({"$or": [{"field1": {"$exists": True}}, {"field2": {"$exists": False}}]}), [{"field%s" % i: i} for i in range(5)]))
    [{'field0': 0}, {'field1': 1}, {'field3': 3}, {'field4': 4}]
    

to_func: Supported operators: Boolean operators

  • $or:

    >>> to_func({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
    "lambda item: ((item['bubu'] > 1) or (item['bubu'] < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    
    >>> to_func({'$or': "invalid value"}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
    >>> list(filter(to_func({'$or': [{"bubu": {"$gt": 3}}, {'bubu': {'$lt': 2}}]}), [{"bubu": i} for i in range(5)]))
    [{'bubu': 0}, {'bubu': 1}, {'bubu': 4}]
    
  • $and:

    >>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}).source
    "lambda item: ((item['bubu'] > 1) and (item['bubu'] < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    >>> to_func({'$or': "invalid value"}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    >>> list(filter(to_func({'$and': [{"bubu": {"$lt": 3}}, {'bubu': {'$gt': 1}}]}), [{"bubu": i} for i in range(5)]))
    [{'bubu': 2}]
    
  • $*nesting*:

    >>> to_func({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]}).source
    "lambda item: ((item['bubu'] > 1) and ((item['bubu'] < 2) or ((item['bubu'] < 3) and (item['bubu'] < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"
    

to_func: Supported operators: Regular expressions

  • $regex:

    >>> to_func({"myfield": {"$regex": 'a'}}).source
    "lambda item, var0=re.compile('a', 0): (var0.search(item['myfield'])) # compiled from {'myfield': {'$regex': 'a'}}"
    
    >>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}).source
    "lambda item, var0=re.compile('a', 2): (var0.search(item['myfield'])) # compiled from {'myfield': {...}}"
    
    >>> to_func({"myfield": {"$regex": 'junk('}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
    
    >>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> import re
    >>> set(re.match(r'(.*): \((.*) and (.*)\)',
    ...     to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, use_arguments=False).source
    ... ).groups()) == {
    ...     'lambda item',
    ...     "(item['myfield'] not in {'aaa'})",
    ...     "(re.search('a', item['myfield'], 0))"
    ... }
    True
    
    >>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
    
    >>> to_func({"bubu": {"$options": "i"}}).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    
    >>> import string
    >>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$options": 'i'}}), [{"myfield": i} for i in string.ascii_letters]))
    [{'myfield': 'a'}, {'myfield': 'b'}, {'myfield': 'c'}, {'myfield': 'A'}, {'myfield': 'B'}, {'myfield': 'C'}]
    
    >>> list(filter(to_func({"myfield": {"$regex": '[a-c]', "$nin": ['c']}}), [{"myfield": i} for i in string.ascii_letters]))
    [{'myfield': 'a'}, {'myfield': 'b'}]
    
    >>> total = len(string.ascii_letters)
    >>> 2 * len(list(filter(
    ...     to_func({"myfield": {"$regex": '[a-z]'}}),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == total
    True
    
    >>> len(list(filter(
    ...     to_func({"myfield": {"$regex": '[a-z]', '$options': 'i'}}),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == total
    True
    
    >>> len(list(filter(
    ...     to_func({"myfield": {"$regex": '[^\d]'}}),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == total
    True
    

to_func (lax mode)

Note

Lax mode allows testing against missing fields.

>>> from mongoql_conv import LaxNone
>>> LaxNone < 1, LaxNone > 1, LaxNone == 0, LaxNone < 0, LaxNone > 0
(False, False, False, False, False)

>>> from mongoql_conv import to_func

>>> to_func({"myfield": 1}, lax=True).source
"lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"

>>> to_func({}, lax=True).source
'lambda item: (True) # compiled from {}'

>>> list(filter(to_func({"bogus": 1}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]

>>> list(filter(to_func({}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

>>> to_func({"myfield": {"$in": [1, 2]}}, lax=True).source
"lambda item, var0={1, 2}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': [1, 2]}}"

>>> list(filter(to_func({"bogus": {"$in": [1, 2]}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[]

>>> to_func({"myfield": {"$in": {1: 2}}}, lax=True).source
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> to_func({"myfield": {"$and": []}}, lax=True).source
"lambda item: (True) # compiled from {'myfield': {'$and': []}}"

>>> list(filter(to_func({"bogus": {"$and": []}}, lax=True), [{"myfield": 1}, {"myfield": 2}]))
[{'myfield': 1}, {'myfield': 2}]

to_func (lax mode): Supported operators

to_func (lax mode): Supported operators: Arithmetic

  • $gt:

    >>> to_func({"myfield": {"$gt": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) > 1) # compiled from {'myfield': {'$gt': 1}}"
    >>> to_func({"myfield": {"$gt": [1]}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part [1]. Expected one of: int,...float, str,...bool, None.
    
    >>> list(filter(to_func({"bogus": {"$gt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $gte:

    >>> to_func({"myfield": {"$gte": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) >= 1) # compiled from {'myfield': {'$gte': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$gte": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $lt:

    >>> to_func({"myfield": {"$lt": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) < 1) # compiled from {'myfield': {'$lt': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$lt": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $lte:

    >>> to_func({"myfield": {"$lte": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) <= 1) # compiled from {'myfield': {'$lte': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$lte": 1}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $eq:

    >>> to_func({"myfield": {"$eq": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': {'$eq': 1}}"
    >>> to_func({"myfield": 1}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) == 1) # compiled from {'myfield': 1}"
    
    >>> list(filter(to_func({"bogus": {"$eq": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $ne:

    >>> to_func({"myfield": {"$ne": 1}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) != 1) # compiled from {'myfield': {'$ne': 1}}"
    
    >>> list(filter(to_func({"bogus": {"$ne": 2}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $mod:

    >>> to_func({"myfield": {"$mod": [2, 1]}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': [2, 1]}}"
    >>> to_func({"myfield": {"$mod": [2, 1, 3]}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part [2, 1, 3]. You must have two items: divisor and remainder.
    
    >>> to_func({"myfield": {"$mod": 2}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 2. Expected one of: list, tuple.
    
    >>> to_func({"myfield": {"$mod": (2, 1)}}, lax=True).source
    "lambda item: (item.get('myfield', LaxNone) % 2 == 1) # compiled from {'myfield': {'$mod': (2, 1)}}"
    
    >>> list(filter(to_func({"bogus": {"$mod": (2, 1)}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    

to_func (lax mode): Supported operators: Containers

  • $in:

    >>> to_func({"myfield": {"$in": (1, 2, 3)}}, lax=True).source
    "lambda item, var0={1, 2, 3}: ('myfield' in item and item.get('myfield', LaxNone) in var0) # compiled from {'myfield': {'$in': (1, 2, 3)}}"
    
    >>> list(filter(to_func({"bogus": {"$in": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(5)]))
    []
    
  • $nin:

    >>> to_func({"myfield": {"$nin": [1, 2, 3]}}, lax=True).source
    "lambda item, var0={1, 2, 3}: ('myfield' not in item or item.get('myfield', LaxNone) not in var0) # compiled from {'myfield': {'$nin': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$nin": {1: 2}}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"bogus": {"$nin": (1, 2, 3)}}, lax=True), [{"myfield": i} for i in range(3)]))
    [{'myfield': 0}, {'myfield': 1}, {'myfield': 2}]
    
  • $size:

    >>> to_func({"myfield": {"$size": 3}}, lax=True).source
    "lambda item: (len(item.get('myfield', LaxNone)) == 3) # compiled from {'myfield': {'$size': 3}}"
    
    >>> to_func({"myfield": {"$size": "3"}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part '3'. Expected one of: int...
    
    >>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": 'x'*i} for i in range(5)]))
    []
    
    >>> list(filter(to_func({"bogus": {"$size": 3}}, lax=True), [{"myfield": list(range(i))} for i in range(5)]))
    []
    
  • $all:

    >>> to_func({"myfield": {"$all": [1, 2, 3]}}, lax=True).source
    "lambda item, var0={1, 2, 3}: (set(item.get('myfield', LaxNone)) >= var0) # compiled from {'myfield': {'$all': [1, 2, 3]}}"
    
    >>> to_func({"myfield": {"$all": 1}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 1. Expected one of: set, list, tuple, frozenset.
    
    >>> list(filter(to_func({"bogus": {"$all": [3, 4]}}, lax=True), [{"myfield": list(range(i))} for i in range(7)]))
    []
    
  • $exists:

    >>> to_func({"myfield": {"$exists": True}}, lax=True).source
    "lambda item: ('myfield' in item) # compiled from {'myfield': {'$exists': True}}"
    
    >>> to_func({"myfield": {"$exists": False}}, lax=True).source
    "lambda item: ('myfield' not in item) # compiled from {'myfield': {'$exists': False}}"
    
    >>> list(filter(to_func({"$or": [{"bogus": {"$exists": True}}]}, lax=True), [{"field%s" % i: i} for i in range(5)]))
    []
    

to_func (lax mode): Supported operators: Boolean operators

  • $or:

    >>> to_func({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
    "lambda item: ((item.get('bubu', LaxNone) > 1) or (item.get('bubu', LaxNone) < 2)) # compiled from {'$or': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    
    >>> to_func({'$or': "invalid value"}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    
    >>> list(filter(to_func({'$or': [{"bogus": {"$gt": 3}}, {'bogus': {'$lt': 2}}]}, lax=True), [{"bubu": i} for i in range(5)]))
    []
    
  • $and:

    >>> to_func({'$and': [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}, lax=True).source
    "lambda item: ((item.get('bubu', LaxNone) > 1) and (item.get('bubu', LaxNone) < 2)) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'bubu': {'$lt': 2}}]}"
    >>> to_func({'$or': "invalid value"}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'invalid value'. Expected one of: list, tuple.
    >>> list(filter(to_func({'$and': [{"bogus": {"$lt": 3}}, {'bogus': {'$gt': 1}}]}, lax=True), [{"bubu": i} for i in range(5)]))
    []
    
  • $*nesting*:

    >>> to_func({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]}, lax=True).source
    "lambda item: ((item.get('bubu', LaxNone) > 1) and ((item.get('bubu', LaxNone) < 2) or ((item.get('bubu', LaxNone) < 3) and (item.get('bubu', LaxNone) < 4)))) # compiled from {'$and': [{'bubu': {'$gt': 1}}, {'$or': [{'bubu': {'$lt': 2}}, {'$and': [{'bubu': {'$lt': 3}}, {'bubu': {'$lt': 4}}]}]}]}"
    

to_func (lax mode): Supported operators: Regular expressions

  • $regex:

    >>> to_func({"myfield": {"$regex": 'a'}}, lax=True).source
    "lambda item, var0=re.compile('a', 0): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {'$regex': 'a'}}"
    
    >>> to_func({"myfield": {"$regex": 'a', "$options": 'i'}}, lax=True).source
    "lambda item, var0=re.compile('a', 2): (var0.search(item.get('myfield', ''))) # compiled from {'myfield': {...}}"
    
    >>> to_func({"myfield": {"$regex": 'junk('}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
    
    >>> to_func({"myfield": {"$regex": 'a', 'junk': 'junk'}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> set(re.match(r'(.*): \((.*) and (.*)\)',
    ...     to_func({"myfield": {"$regex": 'a', '$nin': ['aaa']}}, lax=True, use_arguments=False).source
    ... ).groups()) == {
    ...     "lambda item",
    ...     "(re.search('a', item.get('myfield', ''), 0))",
    ...     "('myfield' not in item or item.get('myfield', LaxNone) not in {'aaa'})"
    ... }
    True
    
    >>> to_func({"bubu": {"$regex": ".*", "$options": "junk"}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'junk'. Unsupported regex option 'j'. Only s, x, m, i are supported !
    
    >>> to_func({"bubu": {"$options": "i"}}, lax=True).source
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    
    >>> import string
    >>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$options": 'i'}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
    []
    
    >>> list(filter(to_func({"bogus": {"$regex": '[a-c]', "$nin": ['c']}}, lax=True), [{"myfield": i} for i in string.ascii_letters]))
    []
    
    >>> total = len(string.ascii_letters)
    >>> 2 * len(list(filter(
    ...     to_func({"bougs": {"$regex": '[a-z]'}}, lax=True),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == 0
    True
    
    >>> len(list(filter(
    ...     to_func({"bogus": {"$regex": '[a-z]', '$options': 'i'}}, lax=True),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == 0
    True
    
    >>> len(list(filter(
    ...     to_func({"bougs": {"$regex": '[^\d]'}}, lax=True),
    ...     [{"myfield": i} for i in string.ascii_letters]
    ... ))) == 0
    True
    

to_Q

Compiles down to a Django Q object tree:

>>> from mongoql_conv.django import to_Q
>>> print(to_Q({"myfield": 1}))
(AND: ('myfield', 1))

>>> print(to_Q({}))
(AND: )

>>> from test_app.models import MyModel
>>> MyModel.objects.clean_and_create([(i, i) for i in range(5)])
>>> list(MyModel.objects.filter(to_Q({"field1": 1})))
[<MyModel: field1=1, field2='1'>]

>>> list(MyModel.objects.filter(to_Q({"field1": 1, "field2": 1})))
[<MyModel: field1=1, field2='1'>]

>>> print(to_Q({"myfield": {"$in": [1, 2]}}))
(AND: ('myfield__in', [1, 2]))

>>> list(MyModel.objects.filter(to_Q({"field1": {"$in": [1, 2]}})))
[<MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>]

>>> print(to_Q({"myfield": {"$in": {1: 2}}}))
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: Invalid query part {1: 2}. Expected one of: set, list, tuple, frozenset.

>>> print(to_Q({"myfield": {"$and": []}}))
(AND: )

>>> list(MyModel.objects.filter(to_Q({"field1": {"$and": []}})))
[<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]

to_Q: Supported operators

to_Q: Supported operators: Arithmetic

  • $gt:

    >>> print(to_Q({"myfield": {"$gt": 1}}))
    (AND: ('myfield__gt', 1))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$gt": 2}})))
    [<MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $gte:

    >>> print(to_Q({"myfield": {"$gte": 1}}))
    (AND: ('myfield__gte', 1))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$gte": 2}})))
    [<MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $lt:

    >>> print(to_Q({"myfield": {"$lt": 1}}))
    (AND: ('myfield__lt', 1))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$lt": 1}})))
    [<MyModel: field1=0, field2='0'>]
    
  • $lte:

    >>> print(to_Q({"myfield": {"$lte": 1}}))
    (AND: ('myfield__lte', 1))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$lte": 1}})))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>]
    
  • $eq:

    >>> print(to_Q({"myfield": {"$eq": 1}}))
    (AND: ('myfield', 1))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": 1})))
    [<MyModel: field1=1, field2='1'>]
    
    >>> print(to_Q({"myfield": 1}))
    (AND: ('myfield', 1))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$eq": 1}})))
    [<MyModel: field1=1, field2='1'>]
    
  • $ne:

    >>> str(to_Q({"myfield": {"$ne": 1}})) in ["(NOT (AND: ('myfield', 1)))", "(AND: (NOT (AND: ('myfield', 1))))"]
    True
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$ne": 1}})))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=2, field2='2'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $mod:

    >>> print(to_Q({"myfield": {"$mod": [2, 1]}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$mod'
    

to_Q: Supported operators: Containers

  • $in:

    >>> print(to_Q({"myfield": {"$in": (1, 2, 3)}}))
    (AND: ('myfield__in', (1, 2, 3)))
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$in": (1, 2)}})))
    [<MyModel: field1=1, field2='1'>, <MyModel: field1=2, field2='2'>]
    
  • $nin:

    >>> str(to_Q({"myfield": {"$nin": [1, 2, 3]}})) in ["(NOT (AND: ('myfield__in', [1, 2, 3])))", "(AND: (NOT (AND: ('myfield__in', [1, 2, 3]))))"]
    True
    
    >>> list(MyModel.objects.filter(to_Q({"field1": {"$nin": (1, 2)}})))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    
  • $size:

    >>> print(to_Q({"myfield": {"$size": 3}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$size'
    
  • $all:

    >>> print(to_Q({"myfield": {"$all": [1, 2, 3]}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$all'
    
  • $exists:

    >>> print(to_Q({"myfield": {"$exists": True}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: DjangoVisitor doesn't support operator '$exists'
    

to_Q: Supported operators: Boolean operators

  • $or:

    >>> print(to_Q({'$or':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
    (OR: ('bubu__gt', 1), ('bubu__lt', 2))
    
    >>> list(MyModel.objects.filter(to_Q({'$or': [{"field1": {"$gt": 3}}, {'field1': {'$lt': 2}}]})))
    [<MyModel: field1=0, field2='0'>, <MyModel: field1=1, field2='1'>, <MyModel: field1=4, field2='4'>]
    
  • $and:

    >>> print(to_Q({'$and':  [{"bubu": {"$gt": 1}}, {'bubu': {'$lt': 2}}]}))
    (AND: ('bubu__gt', 1), ('bubu__lt', 2))
    
    >>> list(MyModel.objects.filter(to_Q({'$and': [{"field1": {"$gt": 1}}, {'field1': {'$lt': 3}}]})))
    [<MyModel: field1=2, field2='2'>]
    
  • $*nesting*:

    >>> print(to_Q({'$and': [
    ...     {"bubu": {"$gt": 1}},
    ...     {'$or': [
    ...         {'bubu': {'$lt': 2}},
    ...         {'$and': [
    ...             {'bubu': {'$lt': 3}},
    ...             {'bubu': {'$lt': 4}},
    ...         ]}
    ...     ]}
    ... ]}))
    (AND: ('bubu__gt', 1), (OR: ('bubu__lt', 2), (AND: ('bubu__lt', 3), ('bubu__lt', 4))))
    
    >>> list(MyModel.objects.filter(to_Q({'$and': [
    ...     {"field1": {"$gt": 1}},
    ...     {'$or': [
    ...         {'field2': {'$lt': 2}},
    ...         {'$and': [
    ...             {'field2': {'$lt': 5}},
    ...             {'field2': {'$gt': 2}},
    ...         ]}
    ...     ]}
    ... ]})))
    [<MyModel: field1=3, field2='3'>, <MyModel: field1=4, field2='4'>]
    

to_Q: Supported operators: Regular expressions

  • $regex:

    >>> print(to_Q({"myfield": {"$regex": 'a'}}))
    (AND: ('myfield__regex', 'a'))
    
    >>> print(to_Q({"myfield": {"$regex": 'a', "$options": 'i'}}))
    (AND: ('myfield__iregex', 'a'))
    
    >>> print(to_Q({"myfield": {"$regex": 'junk('}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid regular expression 'junk(':...
    
    >>> print(to_Q({"myfield": {"$regex": 'a', 'junk': 'junk'}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part "'junk'". You can only have `$options` with `$regex`.
    
    >>> "('myfield__regex', 'a')" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
    True
    >>> "(NOT (AND: ('myfield__in', ['aaa'])))" in str(to_Q({"myfield": {"$regex": 'a', '$nin': ['aaa']}}))
    True
    
    >>> print(to_Q({"bubu": {"$regex": ".*", "$options": "mxs"}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part 'mxs'. Unsupported regex option 'm'. Only i are supported !
    
    >>> print(to_Q({"bubu": {"$options": "i"}}))
    Traceback (most recent call last):
    ...
    mongoql_conv.InvalidQuery: Invalid query part {'$options': 'i'}. Cannot have $options without $regex.
    
    >>> MyModel.objects.clean_and_create([(None, "prefix__"+i) for i in string.ascii_letters])
    >>> list(MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-b]', "$options": 'i'}})))
    [<MyModel: field1=None, field2='prefix__a'>, <MyModel: field1=None, field2='prefix__b'>, <MyModel: field1=None, field2='prefix__A'>, <MyModel: field1=None, field2='prefix__B'>]
    
    >>> list(MyModel.objects.filter(to_Q({"field2": {"$regex": '[a-c]', "$nin": ['prefix__c']}})))
    [<MyModel: field1=None, field2='prefix__a'>, <MyModel: field1=None, field2='prefix__b'>]
    
    >>> total = MyModel.objects.count()
    
    >>> total == 2 * MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]'}})).count()
    True
    
    >>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '__[a-z]', '$options': 'i'}})).count()
    True
    
    >>> total == MyModel.objects.filter(to_Q({"field2": {"$regex": '[^\d]'}})).count()
    True
    

Extending (implementing a custom visitor)

There are few requirements for a visitor. Fist, you need to be able to render boolean $and:

>>> from mongoql_conv import BaseVisitor
>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
...     def validate_foobar(self, value, field_name, context):
...         return value
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
Traceback (most recent call last):
...
TypeError: Can't instantiate abstract class MyVisitor with abstract methods render_and

This is the minimal code to have a custom generator:

>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
...     def render_and(self, parts, field_name, context):
...         return ' & '.join(parts)
...     def validate_foobar(self, value, field_name, context):
...         return value
>>> MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
"foobar(obj['field'], 'test')"

Ofcourse, it won't do much:

>>> MyVisitor('obj').visit({'field': {'$ne': 'test'}})
Traceback (most recent call last):
...
mongoql_conv.InvalidQuery: MyVisitor doesn't support operator '$ne'

Take a look at ExprVisitor too see all the methods you should implement.

Note that the validate method is optional but if you don't implement it you'll get a warning:

>>> import warnings
>>> class MyVisitor(BaseVisitor):
...     def __init__(self, object_name):
...         self.object_name = object_name
...     def visit_foobar(self, value, field_name, context):
...         return "foobar(%s[%r], %r)" % (self.object_name, field_name, value)
...     def render_and(self, parts, field_name, context):
...         return ' & '.join(parts)
>>> with warnings.catch_warnings(record=True) as w:
...     MyVisitor('obj').visit({'field': {'$foobar': 'test'}})
"foobar(obj['field'], 'test')"
>>> len(w)
1
>>> w[0].message
UserWarning('Missing validator validate_foobar in MyVisitor',)
You can’t perform that action at this time.