Orjson in Pydantic V2

[phillipuniverse]

In Pydantic V1, I use orjson for my json_dumps and json_loads:

from pydantic import BaseModel
import orjson

def _orjson_dumps(v: Any, *, default: Any) -> str:
    # orjson.dumps returns bytes, to match standard json.dumps we need to decode
    # comes from https://docs.pydantic.dev/usage/exporting_models/#custom-json-deserialisation
    return orjson.dumps(
        v,
        default=default,
    ).decode()


class MyModel(BaseModel):

    class Config:
        json_dumps = _orjson_dumps
        json_loads = orjson.loads

I am having a little trouble figuring out how to do this in Pydantic V2, or if this is even necessary. With pydantic-core rewritten in Rust and the performance improvements along with it, is using orjson even necessary? Or would the configuration look more like implementing a SchemaSerializer? Thinking something like this in V2, but wondering what I would give up in Pydantic V2 for this somewhat-ham-fisted approach:

from pydantic_core import SchemaSerializer

class OrJsonSerializer(SchemaSerializer):

    def to_json(self, *args, **kwargs):
        return orjson.dumps(...) # implementation not complete, need to understand args/kwargs

    def to_python(self, *args, **kwargs): # real signature unknown
        return orjson.loads(...). # implementation not complete, need to understand args/kwargs

class MyModel(BaseModel):

    __pydantic_serializer__ = OrJsonSerializer()

[adriangb]

With pydantic-core rewritten in Rust and the performance improvements along with it, is using orjson even necessary?

No, using orjson is generally not necessary. I'm not sure that we've benchmarked it but the built-in json dumping/loading should be fast enough.

[shurshilov]

Orjson give very good perfomance for example in FASTAPI 2 codes
This with orjson

And this without


810 vs 583 resuests/second
892 vs 643 MB per 10seconds
~20% speedup
So... how use orjson inside pydantic?
Because if we use this

I think pydantic use slowly serelize by default...
P.S.
tools for testing autocannon 127.0.0.1:81 -d 10 -c 30 -w 3
start fastapi uvicorn 1 worker

[radugrosu]

According to the Migration Guide:

The following config settings have been removed:

• json_loads
• json_dumps

however, no alternative is suggested, and no explanation of why, or to what effect. It seems like a very strange decision, and one that probably puts people off from upgrading from V1.

[samuelcolvin]

Pydantic V2 parses and serialises JSON directly in Rust as an integrated part of validation and marshelling, this is explained at length in many places including the V2 roadmap article.

Because of this we can't and don't need to offer integration with other JSON libraries.

The rate of adoption of pydantic V2 suggests this hasn't put people off, probably because many of them took the time to do some research and think for a second.

[radugrosu]

I'm not usually being told off by maintainers on GitHub, but when I do, I go msgspec.

[arthur-st]

@samuelcolvin It's good to hear that this information is clearly explained in many places, but, unfortunately, it that is not the case for any of the following places:

• Documentation page for JSON
• Documentation page for serialization

[shurshilov]

Pydantic V2 parses and serialises JSON directly in Rust as an integrated part of validation and marshelling, this is explained at length in many places including the V2 roadmap article.

Because of this we can't and don't need to offer integration with other JSON libraries.

The rate of adoption of pydantic V2 suggests this hasn't put people off, probably because many of them took the time to do some research and think for a second.

it doesn't matter to everyone what it is written on, even rust, even c++, as a library user in python, I want speed. Your rust is the same speed as Python json. What is the meaning of it for me as a user? besides the fact that it is written in RUST))kkak it's cool RUST - no)

[shurshilov]

Any highly loaded professional project removes the pydantic as it is slow. What is not clear?

[HansBrende]

@samuelcolvin Python's built-in json.loads in combination with model_validate appears to be roughly 45% faster than model_validate_json. Running on pydantic 2.5.3.

Approach 1: 15 seconds (combined total over 210 server responses)

result = MyModel[MyProps](**json.loads(body_bytes))

Approach 2: 22 seconds (combined total over the same 210 server responses)

result = MyModel[MyProps].model_validate_json(body_bytes)

Both results are better than V1, however, I'm struggling to see the promised speed advantage of model_validate_json... as it seems to be in fact... slower... than json.loads.

[HansBrende]

@samuelcolvin @adriangb update: I think I may have spotted where the discrepancy lies. I have a pre-validator on my Result[P] model. I didn't mention it earlier because it is doing very minimal work and I (incorrectly) assumed it would add the same amount of work to both model_validate and model_validate_json. For the purposes of this discussion, you can assume it's implemented as follows:

class Result(BaseModel, Generic[P]):
    id: int
    uuid: UUID
    properties: P
    
    @model_validator(mode="before")
    @classmethod
    def _pre_validate(cls, values):  # no-op prevalidator
        return values

Yet somehow, whenever I comment this method out, the performance of model_validate_json jumps to be nearly identical to the performance of model_validate(json.loads) (although still significantly slower than model_validate(orjson.loads)). This is surprising to me, since both operations must call this same method if it is defined. I hope this helps!!!

Here are the timings showing the discrepancy between the no-op prevalidator vs. the prevalidator commented out altogether (all run with the GC disabled):

prevalidator commented out:
model_validate + json.loads: 7.3 s
model_validate + orjson.loads: 5.0 s
model_validate_json: 7.6 s

NO-OP prevalidator:
model_validate + json.loads: 7.3 s (same as before)
model_validate + orjson.loads: 5.0 s (same as before)
model_validate_json: 11.2 s (47% worse!!!)

[samuelcolvin]

First of all, it might be worth trying 2.6.0b1 which we released yesterday.

The change in behaviour with a validator is expected - when you add the validator, we have to call to_object() on the JsonValue to construct a python object to pass to the function, that involves iterating over all the data being passed to the function. All subsequent validation is then occurring on the python object created by to_object().

If you have a python validator near the root of your model, you'll loose most of the advantage of model_validate_json, it might even be slower since constructing a JsonValue, then converting that to a python object will always be slower than building a python object directly from JSON as json.loads or orjson do.

You might also try pydantic_core.from_json (as in model.model_validate(pydantic_core.from_json(data))) - from_json isn't yet faster than orjson (although I'm hoping to get there in future), but it's generally faster than json.loads, it also takes a keyword argument cache_strings=False which might improve performance if your data is big enough to bust the string cache that both orjson and pydantic_core.from_json use internally.

[HansBrende]

@samuelcolvin yep, I saw! (Mentioned at the top of my first comment.) All these numbers are from 2.6.0b1 which I pip-installed this morning.

Following your suggestion, I've added pydantic_core.from_json to my timings:

prevalidator commented out:
model_validate + json.loads: 7.3 s
model_validate + orjson.loads: 5.0 s
model_validate_json: 7.6 s
model_validate(pydantic_core.from_json(bytes, cache_strings=True)): 6.3 s
model_validate(pydantic_core.from_json(bytes, cache_strings=False)): 8.2 s

NO-OP prevalidator:
model_validate + json.loads: 7.3 s (same as before)
model_validate + orjson.loads: 5.0 s (same as before)
model_validate_json: 11.2 s (47% worse!!!)
model_validate(pydantic_core.from_json(bytes, cache_strings=True)): 6.4 s
model_validate(pydantic_core.from_json(bytes, cache_strings=False)): 8.3 s

So if validators don't work well with model_validate_json, the docs seem a bit misleading here:

(Or have at least omitted the fact that model_validate(json.loads(...)) is in fact way better if you do have any pre-validators).

Seeing as model_validate(pydantic_core.from_json(bytes)) actually seems to be faster than model_validate_json (in both the cases of either having a pre-validator or not)... I guess this brings me back to my original question of what's the point in combining both json-parsing and validation into one step if separating them into two steps is faster?

[samuelcolvin]

(sorry I didn't see your first comment before, I've now read it)

Good, point we'll update the documentation. I wasn't aware of cases where model_validate(json.loads(...)) was faster than model_validate_json until this conversation. @sydney-runkle maybe we can do a quick PR to update the performance tips page?

I guess this brings me back to my original question of what's the point in combining both json-parsing and validation into one step if separating them into two steps is faster?

1. In many cases it is faster faster than two steps
2. There are numerous types that don't really work with JSON in strict mode data unless you're using model_validate_json - e.g. bytes, datetime, date, timedelta, tuple, set, frozenset, Path - any thing other than the 7 primative types defined by JSON - strict mode was a major new feature of Pydantic V2 that I implemented because lots of people had been asking for it, but it only made sense with builtin JSON parsing
3. This is a journey, with many improvements still to come, in particular disable PyO3 "gil refs" feature pydantic-core#1085 and iterable JSON parsing that we should be able to start working on once disable PyO3 "gil refs" feature pydantic-core#1085 is merged, with those two changes, I'm pretty sure we can reach a point where model_validate_json is pretty much always faster than model_validate(json.loads(...)).

---

By the way, you might want to be a bit careful about using orjson if you're data might have large ints as they're not fully supported - orjson.loads('1' * 20) != int('1' * 20), int parsing should work correctly with json.laods and pydantic_core.from_json.

[HansBrende]

@samuelcolvin Makes sense, thanks for the explanation, and I really appreciate your efforts in this journey! No large integers expected in these payloads, so that shouldn't be a concern. In any case, I've removed orjson anyways in favor of msgspec from this section of my code (for the time being), as my tests indicate that it does all of this (parsing and validation) in a mere 1.5 s (and I figured out a way to hack it to where I don't need a prevalidator whereas it is unclear how to the same in pydantic).¹

As a fellow Rust-enthusiast, I totally understand the complaint about 22K lines of C, however, from a super practical level... I need that speed in this spot. I'm still using pydantic everywhere else... just not in this one critical CPU bottleneck. That being said, I will be keeping an eye on your performance journey! Theoretically, down the line, it should be possible to get that kind of speed in Rust too I would think.

¹_{The msgspec hack which removes the need for a prevalidator is declaring __supertype__: ClassVar[type] = Annotated[str, Meta(extra={'REPLACE_ME': True})] on a certain Struct (the same one I referred to as IdModel) so that msgspec thinks IdModel is actually a NewType around a str for parsing purposes, but PyCharm thinks it's a regular IdModel for typing purposes. Then, in the __post_init__ stage, I simply replace these str values with a corresponding IdModel which I look up elsewhere (don't ask me why 🥴) in the response JSON. I haven't figured out how to do this in pydantic to avoid the prevalidator performance hit, but am open to suggestions and can elaborate if you want more details.}

[PaulRudin]

JOOI - what's the latest benchmarking here? Is the native pydantic (de)serialization significantly slower than orjson?

[biozz]

There is another point, which is kind of discussed here, but not directly - custom encoders.

I have a project, which for the past decade used vanilla json, then ujson, then orjson. This was all before adopting pydantic. There is still quite a lot of legacy code, which uses dumps and loads methods directly. These methods are abstracted into a separate module (from app.utils import json) to be able to change the library for entire project (I know, this is a bold move). I define custom encoders, which were also used in Django models inside JSONField. And for each transition from one library to another, I only had to update how encoders are defined, not the way the libraries are used. It was nice to have json_dumps and json_loads in v1, because I was confident that JSON output would be the same for Pydantic models as it was before introducing Pydantic.

Here are several examples of the encoder rules:

• Decimals are dumped strings
• if Decimal field is None, it becomes Decimal('0.0')
• uuid.UUID are also dumped as strings
• datetimes are dumped as isoformat strings

There are a couple of more rules, but you get the idea.

I understand, that there is now a concept of Serialization, but that isn't quite the same as before, when I could re-use the well established serialization rules. I guess for now I will have to use model.dict() and then use my helper JSON module to serialize the data.

I don't know if I have any specific questions right now, but I just wanted to point out that there is more to the discussion, then performance considerations.

[samuelcolvin]

Thanks for this @biozz with v2, you'll probably want to use model_dump(mode='json') then apply your custom serialization logic.

I don't know if it's faster or slower than model_dump_json(), but in the vast majority of scenarios the performance difference will be negletable.