Handle references within generics of EmbeddedModel

[suconakh]

Bug

When i try to reference other model in EmbeddedModel, i get full document inserted instead of reference to it

Current Behavior

I have these models:

class Need(Model):
    name: str


class PetNeed(EmbeddedModel):
    need: Need = Reference()
    value: int = Field(default=0)


class Pet(Model):
    name: str
    needs: List[PetNeed] = Field(default=[])

If i try to do this:

hunger = Need(name='Hunger')
dog = Pet(name='Dog')
dog.needs.append(PetNeed(need=hunger))
await engine.save_all([hunger, dog])

I get this in the database:

{
    "_id": {
        "$oid": "60062e817a596be396df8ab8"
    },
    "name": "Dog",
    "needs": [{
        "need": {
            "name": "Hunger",
            "id": {
                "$oid": "60062e817a596be396df8ab7"
            }
        },
        "value": 0
    }]
}

As you can see full model being inserted into the database with both name and id instead of just reference id to it

However everything is fine if i reference Need model directly:

rest = Need(name='Rest')
cat = Pet(name='Cat', need=rest)
await engine.save_all([rest, cat])

Database:

{
    "_id": {
        "$oid": "6006326f0092a86615eeae91"
    },
    "name": "Cat",
    "need": {
        "$oid": "6006326f0092a86615eeae90"
    }
}

Expected behavior

ODMantic should put reference id in EmbeddedModel instead of putting full Model in the field

Environment

• ODMantic version: 0.3.2
• MongoDB version: 4.2.11
• Pydantic infos (output of python -c "import pydantic.utils; print(pydantic.utils.version_info())):

pydantic version: 1.7.3
            pydantic compiled: True
                 install path: D:\_Projects\python\bot\py\Lib\site-packages\pydantic
               python version: 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)]
                     platform: Windows-10-10.0.18362-SP0
     optional deps. installed: ['typing-extensions']

Additional context

It is also impossible to make reference fields Optional or put them in a List, but I read in the docs that it is planned to make a generic Reference[T] type so it's ok

[art049]

Hello, yes this is unfortunately not supported yet.
Since #269 you'll get a TypeError when trying a such definition.

A full rework of Reference management is coming soon and this will include this feature 🎉

[Tatayoyoh]

Hi there !

Not exactly the same topic but I managed through Pydantic to solve multi reference problem : get Reference ObjectId into database and full sub Reference data into FastAPI response. This was made through Pydantic custom fields validation.

from database.model_utils import ModelObjectId

class Skill(Model):
    description:str = Field()

class Student(Model):
    class SkillObjectId(ModelObjectId):
        _model = Skill

    fullname:str = Field()
    skills:List[SkillObjectId]   # Many References relation. Here we use a specific type to retrieve data on the fly
    """
    skills:List[ObjectId]        # Many References ODMantic original way
    """

Code here : https://github.com/Tatayoyoh/odmantic-many-relations-model

I hope it can help :)

[realyashnag]

@art049 Is the "full rework of Reference management" still in the works? Any update?