Support .$id and other fields of DBRef objects.

[cypherlou]

Hi MongoMock team.

I've been using your brilliant library for a while and have recently encountered an issue when producing test cases for Beanie with Linked documents.

mongomock==3.23.0
mongomock-motor==0.0.2
beanie==1.8.12

When attempting to create a linked document I get the following error;

    def to_ref(self):
        if self.id is None:
            raise DocumentWasNotSaved("Can not create dbref without id")
>       return DBRef(self.get_motor_collection().name, self.id)
E       AttributeError: 'AsyncIOMotorCollection' object has no attribute 'name'

.env/lib/python3.8/site-packages/beanie/odm/documents.py:1174: AttributeError

I have resolved the problem with the following change to beanie/odm/documents.py;

1094a1095
>         collection_meta.motor_collection.name = collection_meta.name

More explicitly, adding the collection name in a name attribute of collection_meta.motor_collection.

    @classmethod
    def get_motor_collection(cls) -> AsyncIOMotorCollection:
        """
        Get Motor Collection to access low level control

        :return: AsyncIOMotorCollection
        """
        collection_meta = cls.get_settings().collection_settings
        collection_meta.motor_collection.name = collection_meta.name
        return collection_meta.motor_collection

I'm logging the issue with you folks as the error doesn't manifest itself when interacting with an actual mongodb so presumably it isn’t an issue Beanie or the Async engine.

If I'm in the wrong place, please let me know. If you want me to have a dig into the code and PR-it then happy to take a look too - I ask as it might be as long reviewing my PR as making the change and merging it yourselves.

[denivyruck]

Isn't this an issue with mongomock-motor then? Seems they're not copying the name

[cypherlou]

Hey @denivyruck, thanks for the response. I'll pop over and see what the mongomock-motor guys say/think. 👍

[michaelkryukov]

import asyncio
from mongomock_motor import AsyncMongoMockClient
from motor.motor_asyncio import AsyncIOMotorClient
from bson import DBRef

async def test__aggregate_lookup_dbref():
    client = MockOrActualClient('mongodb://localhost:27017')
    await client.db.a.insert_one({
        '_id': 1,
        'refs': [DBRef('b', 2), DBRef('b', 4, 'db')],
    })
    await client.db.b.insert_many([
        {'_id': 2, 'should': 'include'},
        {'_id': 3, 'should': 'skip'},
        {'_id': 4, 'should': 'include'}
    ])
    actual = list(await client.db.a.aggregate([
        {'$lookup': {
            'from': 'b',
            'localField': 'refs.$id',
            'foreignField': '_id',
            'as': 'b'
        }}
    ]).to_list(None))
    assert [{
        '_id': 1,
        'refs': [DBRef('b', 2), DBRef('b', 4, 'db')],
        'b': [
            {'_id': 2, 'should': 'include'},
            {'_id': 4, 'should': 'include'}
        ]
    }] == actual, actual

This test case works with mongo 5, 4 (and fails on 3 because of field name '$id'), but fails with mongomock.

As far as I can tell, mongomock can't properly handle DBRef objects – get_value_by_dot should implement something like:

        elif DBRef and isinstance(result, DBRef):
            if key_item == '$id':
                result = result.id
            elif key_item == '$db':
                result = result.database
            elif key_item == '$ref':
                result = result.collection
            else:
                raise KeyError(key_index)

and _handle_lookup_stage should use get_value_by_dot with can_generate_array=True. I made another test case for singular DBRef – it also works. I didn't check any specifications, but it looks like the proper solution.

[pcorpet]

Hi, I'm not familiar with beanie nor with motor. Given your last comment, the bug seems to be connected to usage of DBRef in mongomock and the use of the magic $id, $db or $ref on it. I'll update the bugs title.