Skip to content

feat(python): support nested field schema encodings#3643

Merged
chaokunyang merged 3 commits intoapache:mainfrom
chaokunyang:refine_nested_container_field_read_for_python
May 2, 2026
Merged

feat(python): support nested field schema encodings#3643
chaokunyang merged 3 commits intoapache:mainfrom
chaokunyang:refine_nested_container_field_read_for_python

Conversation

@chaokunyang
Copy link
Copy Markdown
Collaborator

@chaokunyang chaokunyang commented May 1, 2026

Why?

Python currently loses declared integer encoding aliases inside nested containers, so schemas such as Dict[pyfory.fixed_int32, List[pyfory.tagged_int64]] can fall back to runtime inference or incompatible local assignment behavior. This breaks cross-language schema fidelity for nested map/list/set elements, especially in compatible mode.

What does this PR do?

  • Preserves declared Python field schema metadata for nested container keys, values, and elements in both pure Python and Cython serializers.
  • Extends primitive container fast paths to write/read fixed, varint, tagged, signed, unsigned, bool, and float schema-owned primitive encodings directly.
  • Adds compatible-read assignment planning and validation so remote nested integer encodings are consumed with remote metadata and assigned only when they satisfy the local schema.
  • Updates Python code generation to keep nested integer schema aliases in generic containers instead of converting those lists to numpy array hints.
  • Adds Python unit coverage, compiler generation coverage, and Java/Python xlang tests for signed and unsigned nested annotated containers.
  • Documents nested Python collection aliases and compatible-read behavior in README and Python guide pages.

Related issues

#1017
#3630
#3625
#3630
#3636
#3639
#3640
#3642

AI Contribution Checklist

  • Substantial AI assistance was used in this PR: yes / no
  • If yes, I included a completed AI Contribution Checklist in this PR description and the required AI Usage Disclosure.
  • If yes, my PR description includes the required ai_review summary and screenshot evidence of the final clean AI review results from both fresh reviewers on the current PR diff or current HEAD after the latest code changes.

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change?
  • Does this PR introduce any binary protocol compatibility change?

Benchmark

@chaokunyang chaokunyang merged commit 768dee5 into apache:main May 2, 2026
64 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants