Skip to content

fix: honor msgspec rename in schema_dump + make OffsetPagination runtime-introspectable#421

Merged
cofin merged 10 commits intomainfrom
fix/annotation-thing
Apr 22, 2026
Merged

fix: honor msgspec rename in schema_dump + make OffsetPagination runtime-introspectable#421
cofin merged 10 commits intomainfrom
fix/annotation-thing

Conversation

@cofin
Copy link
Copy Markdown
Member

@cofin cofin commented Apr 22, 2026

Summary

Fixes two silent-divergence bugs between declared schema contract and wire output:

Implementation

#418 — schema_dump rename fix

_dump_msgspec_fields / _dump_msgspec_excluding_unset now iterate msgspec.structs.fields(type(value)) and use field.encode_name for the dict key + field.name for the attribute lookup. Same pattern already established by sqlspec.utils.type_guards.get_msgspec_rename_config.

#419 — OffsetPagination runtime types

Two-layer fix:

  1. Root cause: Convert OffsetPagination to msgspec.Struct(Generic[T]). msgspec's metaclass captures annotations at class-creation time so they survive mypyc, and Litestar natively recognizes msgspec.Struct subclasses for OpenAPI schema generation.

  2. Mypyc-compat split: Wheel validation surfaced TypeError: mypyc classes can't have a metaclass — mypyc refuses to compile any class with a non-default metaclass. OffsetPagination moved to sqlspec/core/_pagination.py (added to the mypyc exclude list) and re-exported from sqlspec.core.filters. The public import path is unchanged. Hot-path filter classes in filters.py continue to compile.

  3. Defense-in-depth: Register _OffsetPaginationSchemaPlugin (an OpenAPISchemaPlugin) in SQLSpecPlugin that emits {items: array<T>, limit, offset, total} explicitly — guards against any future Litestar/msgspec drift that would break auto-detection.

  4. Cleanup: Delete the now-redundant _encode_offset_pagination type_encoder — msgspec.Struct is encoded natively by Litestar's default serialization to the same JSON shape.

Behavior changes

Converting OffsetPagination to msgspec.Struct changes three dunder behaviors (called out in docs/changelog.rst):

  • __eq__ is now field-wise (was identity)
  • __hash__ is None (msgspec default for non-frozen Structs — was id-based)
  • __repr__ is informative OffsetPagination(items=..., limit=..., offset=..., total=...)

Test plan

  • uv run pytest tests/unit/ tests/integration/extensions/litestar/ --ignore=tests/unit/adapters — 4423 passed, 26 skipped, 0 failed
  • uv run ruff check sqlspec tests — all checks passed
  • Red phase confirmed: 5/6 rename tests fail on snake_case keys before fix; OpenAPI test fails with Item not in OpenAPI components: set() before fix
  • Green phase confirmed: every task passes tests immediately after the matching fix
  • HATCH_BUILD_HOOKS_ENABLE=1 uv build --wheel — compiled wheel produced filters.cpython-310-x86_64-linux-gnu.so (compiled) + _pagination.py (uncompiled)
  • Issue OffsetPagination has empty runtime __annotations__ — Litestar OpenAPI schema generator produces empty schema for paginated responses #419 reproducer against installed compiled wheel prints components: ['Item', 'OffsetPagination___main__.Item_'] and OK: Item registered, response schema non-empty
  • Docs recipes spot-check — docs/recipes/service_layer.rst uses keyword-arg OffsetPagination(items=..., limit=..., offset=..., total=...) construction, wire-compatible with msgspec.Struct, no changes needed

Commits

8 commits on fix/annotation-thing, Red-Green-Refactor per chapter:

284237d4 docs(changelog): add schema wire correctness release notes
33186c86 refactor(core): move OffsetPagination to _pagination.py (mypyc compat)
774287a3 feat(litestar): register _OffsetPaginationSchemaPlugin for OpenAPI
ef9c1366 refactor(litestar): remove dead _encode_offset_pagination
93dc6c9f fix(filters): convert OffsetPagination to msgspec.Struct
209e13c6 test(litestar): add failing OpenAPI schema assertion
4c1aad07 fix(serializers): honor msgspec rename meta in schema_dump
3cf93b3f test(serializers): add failing rename matrix for schema_dump

Fixes #418
Fixes #419

cofin added 8 commits April 22, 2026 16:12
6-case test class (camel/kebab/pascal/callable/no-rename/exclude_unset+rename)
asserting that schema_dump honors msgspec.Struct rename meta. 5/6 fail on
current main because _dump_msgspec_fields iterates __struct_fields__ (attribute
names) instead of structs.fields().encode_name.

Beads: sqlspec-p9d.1.1
Refs: #418
Switch _dump_msgspec_fields and _dump_msgspec_excluding_unset from iterating
value.__struct_fields__ (Python attribute names) to msgspec.structs.fields(),
using field.encode_name for the dict key and field.name for attribute lookup.

Structs declared with rename="camel"/"kebab"/"pascal"/callable now emit the
renamed wire names. Structs without rename meta are unchanged (encode_name
equals name). Matches the precedent already set by
sqlspec.utils.type_guards.get_msgspec_rename_config.

Beads: sqlspec-p9d.1.2
Fixes #418
…tion (#419)

Asserts that OffsetPagination[Item] registers Item as an OpenAPI component
and emits a non-empty response schema. Fails on current code because
OffsetPagination is a plain Generic[T] class that Litestar cannot introspect.

Beads: sqlspec-p9d.2.1
Refs: #419
The previous plain-class Generic[T] shape was invisible to Litestar's OpenAPI
generator (no introspection adapter) and lost runtime __annotations__ under
mypyc compilation. Converting to msgspec.Struct fixes both: msgspec's
metaclass preserves annotations through mypyc, and Litestar natively
introspects msgspec.Struct subclasses for OpenAPI schema generation.

Positional ctor signature unchanged (items, limit, offset, total). Wire
format unchanged (msgspec encodes to the same JSON shape the former custom
type_encoder produced).

Behavior deltas (to be release-noted in C2.T6):
- __eq__: identity -> field-wise (improvement)
- __hash__: id-based -> unhashable (frozen=False default)
- __repr__: default object repr -> OffsetPagination(items=..., ...) (improvement)

Beads: sqlspec-p9d.2.2
Refs: #419
OffsetPagination is now a msgspec.Struct (93dc6c9) and encodes natively
through Litestar's default serialization. The custom type_encoder and its
registration block are dead code.

Integration test test_litestar_offset_pagination_serialization continues to
pass — proves native encoding produces the same JSON shape.

OffsetPagination import is dropped here and re-added in C2.T4 when
_OffsetPaginationSchemaPlugin is introduced.

Beads: sqlspec-p9d.2.3
Refs: #419
)

Defense-in-depth: registers an OpenAPISchemaPlugin that expands
OffsetPagination[T] into an explicit {items: array<T>, limit, offset, total}
Schema object. Guards against any future drift in Litestar or msgspec that
could break the native msgspec.Struct auto-detection installed in 93dc6c9.

Integration test strengthened to assert the required/properties sets match
{items, limit, offset, total} — proves the plugin fires and produces the
canonical shape.

Beads: sqlspec-p9d.2.4
Refs: #419
#419)

Discovered during wheel validation (C2.T5): mypyc refuses to compile any
class that declares a metaclass, and msgspec.Struct uses one. Colocating
OffsetPagination as msgspec.Struct inside the mypyc-compiled
sqlspec/core/filters.py breaks wheel builds with:

  TypeError: mypyc classes can't have a metaclass

Fix: split OffsetPagination into a sibling module sqlspec/core/_pagination.py
and add that file to the mypyc exclude list. Re-export from
sqlspec.core.filters so the public import path is unchanged. The hot-path
filter classes (StatementFilter, BeforeAfterFilter, etc.) remain in
filters.py and continue to be compiled.

Also switched the OpenAPISchemaPlugin from string 'array'/'object'/'integer'
Schema types to litestar.openapi.spec.OpenAPIType enum members to satisfy
Litestar's type signature.

Verified: compiled wheel loads, OffsetPagination.__annotations__ is
non-empty, #419 reproducer script exits with 'OK: Item registered, response
schema non-empty'.

Beads: sqlspec-p9d.2.2, sqlspec-p9d.2.5
Refs: #419
Documents the two fixes shipped in this PR:
- schema_dump now honors msgspec rename meta
- OffsetPagination is a msgspec.Struct (lives in _pagination.py for
  mypyc compatibility) and its annotations survive compiled wheels

Also flags the three ABI deltas from the OffsetPagination conversion
(field-wise __eq__, unhashable, informative __repr__).

Beads: sqlspec-p9d.2.6
Refs: #418, #419
Copy link
Copy Markdown

@wpuziewicz wpuziewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cofin added 2 commits April 22, 2026 16:59
CI surfaced three failures on PR #421:

1. MyPyC wheel build — 'ModuleNotFoundError: No module named msgspec' when
   importing sqlspec in the cibuildwheel test environment. msgspec is an
   OPTIONAL dep (pyproject.toml:51). The previous commit introduced a
   module-level 'import msgspec' in sqlspec/core/_pagination.py and
   sqlspec/utils/serializers/_schema.py, which unconditionally required
   msgspec just to import sqlspec.

   Fix:
   - _pagination.py: wrap the msgspec.Struct definition in try/except,
     fall back to a plain Generic[T] container when msgspec is unavailable.
   - _schema.py: move 'from msgspec import structs' inside the two
     _dump_msgspec_* functions (matches existing lazy-import pattern at
     sqlspec/utils/type_guards.py).

2. mypy — 'Value of type dict[str, PathItem] | None is not indexable' on
   schema.paths[...] access in the integration test. Cast schema.paths to
   dict[str, Any] and assert schema.components.schemas is not None.

3. pyright — same class of Optional/union issues resolved by the same
   casts.

Verified locally:
- uv run mypy: 1023 source files, no issues
- uv run pyright on the test file: 0 errors
- 8/8 affected tests pass

Refs: #418, #419
…uct (#419)

msgspec is an OPTIONAL sqlspec dependency; the previous msgspec.Struct
conversion regressed that invariant. Switching to a stdlib @DataClass:

1. Removes the msgspec runtime requirement — sqlspec imports cleanly on
   msgspec-free installs.
2. Still delivers runtime-introspectable __annotations__ (the root fix
   for #419) — Python stores @DataClass field annotations on the class
   dict, and the module is kept out of the mypyc compile set.
3. Litestar's OpenAPI generator natively recognizes @DataClass types, so
   OffsetPagination[T] still produces a component schema referencing T.
4. Litestar's default serialization (msgspec-backed when installed, stdlib
   fallback otherwise) still emits {items, limit, offset, total}.

mypyc exclude is retained: @DataClass mutates the class at definition
time, which mypyc-compiled classes forbid ('AttributeError: __dict__ of
type is not writable'). The pyproject comment is updated to reflect the
new reason.

Behavior deltas from the conversion (from PLAIN-CLASS before) remain:
- __eq__ is now field-wise
- __hash__ is None (non-frozen dataclass default)
- __repr__ is informative OffsetPagination(...)

Changelog updated to describe the dataclass path (no msgspec.Struct
reference) and explain why msgspec is not required.

Verified against built compiled wheel:
- Install without msgspec: sqlspec imports, OffsetPagination has hints,
  dataclass detection works.
- Install with msgspec + litestar: OpenAPI registers Item component,
  wire JSON shape unchanged.

Refs: #419
@cofin cofin merged commit 3ebce0f into main Apr 22, 2026
16 checks passed
@cofin cofin deleted the fix/annotation-thing branch April 22, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants