Document differences between structs.asdict/astuple and to_builtins#1025
Open
Document differences between structs.asdict/astuple and to_builtins#1025
Conversation
Add a usage docs section explaining that to_builtins is the encoding half (respects omit_defaults, UNSET, and recurses into nested struct-like types) while structs.asdict/astuple perform a one-to-one conversion following the dataclasses.asdict/astuple contract. Cross-reference the new section from the omit_defaults and UNSET sections, and update the C docstrings for asdict, astuple, and to_builtins. Fixes #778. Supersedes #780.
The previous wording only called out omit_defaults and UNSET, but to_builtins differs from asdict/astuple in more ways: rename, tag injection, array_like, recursion into nested Struct/dataclass/attrs/ TypedDict/NamedTuple values, and all the value-level conversions (bytes to base64, datetime to ISO 8601, UUID/Decimal to string, set to list, Enum to member value). This matters because prior confused-user reports cover all of these axes: #748 tripped on rename, #830 on array_like plus recursion.
The msgspec docs use default_role='obj', so single-backtick references like `msgspec.to_builtins` and `dataclasses.dataclass` auto-link to the api page or (via intersphinx) to the Python stdlib docs. Switch Python-object references from double backticks (literal) to single backticks so they link, and use fully-qualified names so they resolve. Double backticks are kept for kwarg names and setting values that aren't importable objects (rename, tag, omit_defaults, array_like, enc_hook, str_keys, order, builtin_types). Also add an attrs_ target in usage.rst for the attrs library link.
- Unify the order of Struct-level settings across docstrings and usage docs to `rename, omit_defaults, array_like, tag`. - Split the 60-word sentence at the end of the usage section into three shorter paragraphs (what asdict/astuple do, what they don't do, when to prefer to_builtins). - Condense the value-level conversions bullet in the `to_builtins` docstring; the full list now lives in the usage docs via a cross-reference. - Soften the closing sentence of the `to_builtins` docstring: 'do none of this' -> 'perform a plain one-to-one conversion, applying none of the above'. - In the usage bullet list, split 'struct-level settings' and 'UNSET omission' into separate bullets so UNSET isn't lumped in with settings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Clarify how
msgspec.structs.asdict/msgspec.structs.astuplediffer frommsgspec.to_builtins. The three functions all produce "plain Python" output from aStruct, but they serve different roles and apply different settings:structs.asdict/structs.astuple: Python-level, modeled ondataclasses.asdict/dataclasses.astuple. One-to-one conversion, no recursion into nested struct-like types,omit_defaultsandUNSETare ignored.to_builtins: part of the encoder pipeline. Appliesomit_defaults, omitsUNSETfields, recursively converts nestedStruct/dataclass/attrs/TypedDict/NamedTuplevalues, and appliesenc_hook/str_keys/order.The existing
to_builtinsdocstring said it is "mainly useful for adding msgspec support for other protocols", which underplays its role when partial-processing a message (e.g. handing a dict tohttpx, which is the motivating case in #778).Motivation
Users repeatedly trip over this distinction, reach for
asdictas the natural API, and are surprised that encoder settings don't apply. The discoverability ofto_builtinsas the right tool is also low. A non-exhaustive sample of closed issues:to_builtinsfunction converts aStructobject without default values #748 - user expectedasdictto honoromit_defaults. Self-closed asNOT_PLANNEDafter another user explained thatto_builtinsis what they wanted.asdictto apply struct settings. Self-closed asCOMPLETEDwith a shrug: "Probably it was intended because it makes some sense."array_likeStructsasdict#830 - user expectedasdictto recurse into nestedarray_likestructs. Foundto_builtinsas a workaround.omit_defaultshandling inasdictandto_builtinsdocumentation #778 - current issue, same root cause, explicitly asking for docs (ncoghlan).Behavior-only change would be a breaking change to
asdict/astuple, and #748 / #817 both confirmed the current behavior is intentional. So this PR is docs-only: make the boundary explicit so users stop bouncing offasdictand ontoto_builtinsvia stackoverflow / github search.Changes
docs/usage.rst: newConverting to and from Builtin Typessection that pairsto_builtins/convertwithencode/decode, and contrasts them withstructs.asdict/astuple. Also adds ato-builtins-vs-asdictcross-reference label.docs/structs.rst:omit_defaultssection now notes that the setting only affects encoders andto_builtins.docs/supported-types.rst:UNSETsection now notes the same forUNSETomission.src/msgspec/_core.c: docstrings forstructs.asdict,structs.astuple, andto_builtinsupdated to state these semantics explicitly and to cross-link each other.Notes
omit_defaultsandUNSETcross-references, updated docstrings forastupleas well, and recursion-into-nested-structs is called out. Paths also rebased (Cover conversion to/from builtins in usage docs #780 still targeteddocs/source/andmsgspec/_core.c, both since moved).Fixes #778.