The misuse of smart union in pydantic

[numb3r3]

The string value "1" in doc.tags will be automatically converted to bool True when using .to_pydantic_model. This would not make sense.

The reproduce script:

from docarray import Document

doc = Document(tags={'a': '1', 'b': 1, 'c': '2'})
print(f'origin:')
print(doc.tags)

pd_doc_tags = doc.to_pydantic_model().tags
print(f'pydantic: ')
print(pd_doc_tags)


proto_tags = doc.to_dict(protocol='protobuf')['tags']
print(f'protobuf: ')
print(proto_tags)


json_tags = doc.to_dict(protocol='jsonschema')['tags']
print(f'jsonschema')
print(proto_tags)

yields the following output:

origin:
{'a': '1', 'b': 1, 'c': '2'}
pydantic: 
{'a': True, 'b': True, 'c': 2.0}
protobuf: 
{'a': '1', 'c': '2', 'b': 1.0}
jsonschema
{'a': '1', 'c': '2', 'b': 1.0}

[hanxiao]

This to be honest is an upstream issue, i don't think we can solve it?

[numb3r3]

One possible quick solution is to disable the smart union in the pydantic model definition

class PydanticDocument(BaseModel):
    ...
    tags: Optional[Dict[str, '_StructValueType']] -> tags: Optional[Dict[str, ANY]]

https://github.com/jina-ai/docarray/blob/main/docarray/document/pydantic_model.py#L30-L50

But I'm not very clear whether this change will not destroy other API/functions.

[numb3r3]

With this PR #73, we can get the following result

origin:
{'a': '1', 'b': 1, 'c': '2'}
pydantic: 
{'a': '1', 'b': 1, 'c': '2'}
protobuf: 
{'b': 1.0, 'c': '2', 'a': '1'}
jsonschema
{'b': 1.0, 'c': '2', 'a': '1'}