Add support for Generic Structs

[jcrist]

This adds full support for encoding/decoding generic msgspec.Struct types. Fixes #193.

A quick demo:

from typing import Generic, TypeVar

from msgspec import Struct, json, ValidationError

T = TypeVar("T")

class Paginated(Struct, Generic[T]):
    page: int
    per_page: int
    total_count: int
    items: list[T]


class Order(Struct):
    item: str
    count: int


msg = """
{
    "page": 2,
    "per_page": 5,
    "total_count": 99,
    "items": [
        {"item": "apple", "count": 3},
        {"item": "banana", "count": 2},
        {"item": "carrot", "count": 10},
        {"item": "durian", "count": 1},
        {"item": "eggplant", "count": 4}
    ]
}
"""

# Decoding uses the parametrized type to validate
print(json.decode(msg, type=Paginated[Order]))
#> Paginated(
#>     page=2,
#>     per_page=5,
#>     total_count=99,
#>     items=[
#>         Order(item='apple', count=3),
#>         Order(item='banana', count=2),
#>         Order(item='carrot', count=10),
#>         Order(item='durian', count=1),
#>         Order(item='eggplant', count=4)
#>     ]
#> )

# A validation error is raised if the message doesn't match
bad_msg = """
{
    "page": 1,
    "per_page": 1,
    "total_count": 1,
    "items": [
        {"item": "apple", "count": "oops"}
    ]
}
"""
try:
    json.decode(bad_msg, type=Paginated[Order])
except ValidationError as err:
    print(err)
#> ValidationError("Expected `int`, got `str` - at `$.items[0].count`")

Recursive and complicated parametrizations also should work, including the weird challenge posted on the pydantic twitter:

from __future__ import annotations
from typing import Generic, TypeVar

import msgspec

T = TypeVar("T")

class MyGenericModel(msgspec.Struct, Generic[T]):
    foobar: list[MyGenericModel[T]] | None
    spam: T


MyGenericModelList = MyGenericModel[list[T]]

class MyGenericModelIntList(MyGenericModelList[int]):
    pass

msg = b'{"foobar": [{"foobar": null, "spam": [1]}], "spam": [1, 2, 3]}'

m = msgspec.json.decode(msg, type=MyGenericModelIntList)
print(m)
#> MyGenericModelIntList(
#>   foobar=[MyGenericModel(foobar=None, spam=[1])],
#>   spam=[1, 2, 3]
#> )

In common usage, generics should have no runtime overhead. Since msgspec doesn't validate on __init__ we don't need to generate a new type for each parametrization - type type of Paginated[Order] is typing._GenericAlias. No magic on our side needed!

The scoping rules and behavior for handling complicated parametrized generics are underdocumented in the corresponding PEPs. I believe we've covered all the edge cases here, but would still love for a few users to try this out before it's released. If this passes tests I plan on merging it early, but would hope someone could try this out before it's released.

[jcrist]

Alright, things are passing now, merging 🚀.

I hope to cut a release containing generics support in the first week of May. If anyone is interested, I'd love it if someone could install from the main branch (https://jcristharif.com/msgspec/install.html#installing-from-github) and try things out. I believe our test suite should cover all the possible edge cases, but if there are bugs it'd be nice to catch 'em now before the release.

[jcrist]

^^^ cc @provinzkraut since you mentioned you were interested in generic support here, and generally provide excellent feedback :).