Skip to content

[Python] add float16 support using numpy.float16 #3282

@chaokunyang

Description

@chaokunyang

Feature Request

Add float16 support to Fory Python runtime and codegen using numpy.float16 as the public scalar type, plus array support for efficient serialization.

Is your feature request related to a problem? Please describe

We want to use float16 in FDL to reduce payload size and memory footprint, and to interoperate with ML/graphics ecosystems where half precision is common. Python currently lacks a first-class float16 primitive in Fory.

Describe the solution you'd like

1) FDL / Type System

  • Introduce float16 as a new primitive type in FDL.
  • Allow float16 in message fields, repeated fields, map values, and unions (where primitives are allowed).

2) Wire Format / Serialization Semantics

  • Encode float16 as 2 bytes representing the raw IEEE 754 binary16 bit pattern.
  • Endianness must match existing float32/float64 behavior.
  • NaN/Inf/±0/subnormal must round-trip correctly at the bit level (document NaN policy if canonicalized).

3) Python Runtime (core requirement)

Use numpy.float16 as the public scalar type.

3.1 Type definition
  • Public APIs should accept/return numpy.float16 (not raw uint16 bits).
  • No custom float16 class should be required.
3.2 Arrays / float16_array
  • For repeated float16, support numpy.ndarray with dtype=float16 as a fast path.
  • Define a float16_array container as a packed array of 16-bit values and map it to numpy.ndarray (dtype=float16).
3.3 Serialization boundary
  • Serializer/deserializer should read/write raw 16-bit patterns from numpy.float16 values and arrays.
  • Avoid per-element Python overhead when handling arrays.

4) Codegen requirement (Python)

  • Generated fields for float16 should use numpy.float16 in type hints.
  • Generated fields for float16_array or repeated float16 should use numpy.ndarray (dtype=float16) where possible.

5) Compiler/Reflection integration

  • Ensure reflection/dynamic serialization recognizes numpy.float16 as the float16 primitive.

6) Tests

  • Conversion and round-trip tests for scalars (±0, ±Inf, NaN, subnormals, max/min normal).
  • Array serialization tests using numpy.ndarray.
  • Cross-language golden tests can be added later.

Describe alternatives you've considered

  1. Use a custom Python Float16 class.
  • Rejected: numpy.float16 is the de-facto standard and already optimized.

Additional context

This issue is the Python counterpart of the per-language float16 work.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions