Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add msgspec.to_builtins method #258

Merged
merged 5 commits into from Jan 10, 2023
Merged

Add msgspec.to_builtins method #258

merged 5 commits into from Jan 10, 2023

Conversation

jcrist
Copy link
Owner

@jcrist jcrist commented Jan 8, 2023

This adds a new msgspec.to_builtins function, for transforming objects composed of all the types msgspec supports to objects composed only of types standard python serializers support (list, tuple, int, str, ...).

While this method may be useful directly to users, the primary intended use is for wrapping it with additional serialization APIs within msgspec itself (for example, a new msgspec.yaml module). While JSON may require high-performance and a builtin encoder, config-files like YAML, TOML, ... usually aren't perforamnce sensitive, so the extra copies here shouldn't matter.

Once this is in, we'll follow up with a msgspec.from_builtins for handling the "decoding" side of things.

This adds a new `msgspec.to_builtins` function, for transforming objects
composed of all the types `msgspec` supports to objects composed only of
types standard python serializers support (`list`, `tuple`, `int`,
`str`, ...).

While this method *may* be useful directly to users, the primary
intended use is for wrapping it with additional serialization APIs
within `msgspec` itself (for example, a new `msgspec.yaml` module).
While JSON may require high-performance and a builtin encoder,
config-files like YAML, TOML, ... usually aren't performance sensitive,
so the extra copies here shouldn't matter.

Once this is in, we'll follow up with a `msgspec.from_builtins` for
handling the "decoding" side of things.
Rather than roundtripping extra config through JSON, we can do the
conversion from rich types -> builtin types directly.
@jcrist jcrist changed the title [WIP] Add msgspec.to_builtins method Add msgspec.to_builtins method Jan 10, 2023
@jcrist
Copy link
Owner Author

jcrist commented Jan 10, 2023

Example of usage:

In [1]: import msgspec                         

In [2]: class User(msgspec.Struct):
   ...:     name: str                          
   ...:     groups: set[str] = set()
   ...:     email: str | None = None
   ...:                                        

In [3]: alice = User("alice", groups={"admin", "engineering"}, email="alice@company.com")

In [4]: alice                                  
Out[4]: User(name='alice', groups={'engineering', 'admin'}, email='alice@company.com')

In [5]: msg = msgspec.to_builtins(alice)  # convert to simple types

In [6]: msg                                    
Out[6]:                                        
{'name': 'alice',                              
 'groups': ['engineering', 'admin'],
 'email': 'alice@company.com'}

In [7]: import yaml                            

In [8]: print(yaml.safe_dump(msg, sort_keys=False))  # can pass to downstream libraries
name: alice                                    
groups:                                        
- engineering                                  
- admin                                        
email: alice@company.com

Like I said above, ideally most users won't ever need to directly access this API - we should instead use it to wrap common downstream libraries (yaml, toml, ...) in high-level APIs mirroring the existing msgspec.json/msgspec.msgpack modules.

@jcrist jcrist merged commit 624e1a1 into main Jan 10, 2023
@jcrist jcrist deleted the to-builtins branch January 10, 2023 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant