How to ensure_ascii in Pydantic v2

[FyZzyss]

Discussed in #8821

^{Originally posted by FyZzyss February 15, 2024}
How to ensure_ascii in Pydantic v2?

SomeModel.model_dump_json() in V2 not ensure ascii symbols anymore.

Example:

class TextMessage(BaseModel):
    text: str

print(TextMessage.model_validate({"text": "Что"}).model_dump_json(by_alias=True, exclude_unset=True))

[sydney-runkle]

@FyZzyss,

Thanks for your question!

You could do something like this:

from pydantic import BaseModel, ConfigDict
import json

class TextMessage(BaseModel):
    text: str


dumped_data = TextMessage.model_validate({"text": "Что"}).model_dump(by_alias=True, exclude_unset=True)
print(dumped_data)
#> {'text': 'Что'}
print(json.dumps(dumped_data, ensure_ascii=True))
#> {"text": "\u0427\u0442\u043e"}

Or even:

from pydantic import BaseModel, model_serializer
import json

class TextMessage(BaseModel):
    text: str

    @model_serializer(mode='wrap', when_used='json')
    def serialize(self, handler) -> str:
        return json.dumps(handler(self), ensure_ascii=True)


print(TextMessage.model_validate({"text": "Что"}).model_dump_json(by_alias=True, exclude_unset=True))
#> "{\"text\": \"\\u0427\\u0442\\u043e\"}"

By default, ensure_ascii is set to false :). Let me know if you have any follow up questions!

[FyZzyss]

@sydney-runkle Thank you for quick answer. Are you planning to return this functionality?

Now it produces more boilerplate code and built-in json module is very slow, so I must import third-party serializers(

[sydney-runkle]

@FyZzyss,

At the moment, we're not planning on adding this functionality - the performance isn't slower than it would be in V1, where we just had a catch-all **kwargs that passed those values onto json.dumps.

We could consider adding support for flags like this on a case by case basis, though I'm not sure how high the demand is for this specific flag. Thanks for following up!

[HansBambel]

This gave me some headache as well! I was using json.dumps before and wanted to use the sleeker in-built functionality from pydantic, but then the input from German clients that contained Umlaute such as "ä", "ö", or "ü" where not converted any more.

Now I have to use import json along with json.dumps again :(

[modrozd]

Also a headache for me, this simple line:
data = product.model_dump_json()
has to be now:
data = json.dumps(json.loads(product.model_dump_json()))

maybe its worth adding it as a non-default option to ensure_ascii?

[sydney-runkle]

Seems like ensure_ascii is a popular flag - if someone wants to open a new feature request, that'd be welcome.

This will probably be quite an involved change bc it'd require changes in jiter and pydantic-core, most likely.