Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dec_hook to Decoder/decode #33

Merged
merged 6 commits into from Jun 27, 2021
Merged

Add dec_hook to Decoder/decode #33

merged 6 commits into from Jun 27, 2021

Conversation

jcrist
Copy link
Owner

@jcrist jcrist commented Jun 22, 2021

Adds a new dec_hook callback supporting type conversions between
builtin msgpack types and arbitrary python types. This hook should have
the following signature:

def dec_hook(obj: Any, type: Type) -> Any:
    ...

This receives an object composed of builtin msgpack types (e.g. dict,
list, int, ...) and the expected decode type (as described by the
decoder's type parameter). It should attempt to convert the value to the
provided type, or raise a type error appropriately. When combined with
enc_hook on the encoder, this lets users serialize arbitrary custom
objects without resorting to extension types.

For example, to serialize/deserialize a NamedTuple you might write
the following:

import msgspec
from typing import NamedTuple

class Point(NamedTuple):
    x: int
    y: int

def enc_hook(obj):
    if isinstance(obj, tuple):
        # convert the named tuple to a supported type
        # in this case it's a tuple like `(1, 2)`
        return tuple(obj)
    raise TypeError(f"Type {type(obj).__name__} is not supported")

def dec_hook(obj, type):
    if issubclass(type, tuple):
        # convert the builtin type (e.g. `[1, 2]`) to
        # the custom type (`Point(x=1, y=2)`)
        return type(*obj)
    raise TypeError(f"Type {type.__name__} is not supported")

enc = msgspec.Encoder(enc_hook=enc_hook)
dec = msgspec.Decoder(Point, dec_hook=dec_hook)

msg = enc.encode(Point(1, 2))
x = dec.decode(msg)
print(x)  # Point(x=1, y=2)

There's still a few TODO items, but this is mostly functional already

  • Tests
    - [ ] Docs
  • Better support GenericAlias objects for custom types

Adds a new `dec_hook` callback supporting type conversions between
builtin msgpack types and arbitrary python types. This hook should have
the following signature:

```python
def dec_hook(obj: Any, type: Type) -> Any:
    ...
```

This receives an object composed of builtin msgpack types (e.g. dict,
list, int, ...) and the expected decode type (as described by the
decoder's type parameter). It should attempt to convert the value to the
provided type, or raise a type error appropriately. When combined with
`enc_hook` on the encoder, this lets users serialize arbitrary custom
objects without resorting to extension types.

For example, to serialize/deserialize a `NamedTuple` you might write
the following:

```python
import msgspec
from typing import NamedTuple

class Point(NamedTuple):
    x: int
    y: int

def enc_hook(obj):
    if isinstance(obj, tuple):
        # convert the named tuple to a supported type
        # in this case it's a tuple like `(1, 2)`
        return tuple(obj)
    raise TypeError(f"Type {type(obj).__name__} is not supported")

def dec_hook(obj, type):
    if issubclass(type, tuple):
        # convert the builtin type (e.g. `[1, 2]`) to
        # the custom type (`Point(x=1, y=2)`)
        return type(*obj)
    raise TypeError(f"Type {type.__name__} is not supported")

enc = msgspec.Encoder(enc_hook=enc_hook)
dec = msgspec.Decoder(Point, dec_hook=dec_hook)

msg = enc.encode(Point(1, 2))
x = dec.decode(msg)
print(x)  # Point(x=1, y=2)
```
msgspec/core.c Outdated Show resolved Hide resolved
@jcrist
Copy link
Owner Author

jcrist commented Jun 24, 2021

TODO: switch the order of parameters in dec_hook. Since the user will most likely be dispatching on the type, it'd be nice if this parameter was first to mirror the convention in other dispatch decorators (e.g. functools.singledispatch).

Unparametrized custom generic types weren't properly normalized
previously, now fixed.

Also squashes a few compiler warnings in python 3.8 for functions taking
no parameters.
@jcrist jcrist changed the title [WIP] Add dec_hook to Decoder/decode Add dec_hook to Decoder/decode Jun 27, 2021
@jcrist
Copy link
Owner Author

jcrist commented Jun 27, 2021

Putting off writing docs til a later PR. This is good for now. Merging.

@jcrist jcrist merged commit 4cebab7 into master Jun 27, 2021
@jcrist jcrist deleted the decode-hook branch June 27, 2021 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant