Skip to content

[JavaScript] add bfloat16 and bfloat16_array #3288

@chaokunyang

Description

@chaokunyang

Feature Request

Add bfloat16 and bfloat16_array support to Fory JavaScript runtime and codegen, following the same structure as the existing float16 work.

Related float16 issue: #3210 (open)

Is your feature request related to a problem? Please describe

We want to use bfloat16 (BF16) in FDL to reduce payload size while keeping a wide exponent range (common in ML/AI workflows). Fory currently lacks a BF16 primitive and optimized arrays.

Describe the solution you'd like

1) FDL / Type System

  • Introduce a new primitive type: bfloat16.
  • Add bfloat16_array as a packed array type (or treat it as a canonical optimized form of repeated bfloat16).
  • Allow bfloat16 in message fields, repeated fields, map values, and unions (where primitives are allowed).

2) Wire Format / Serialization Semantics

  • Encode bfloat16 as 2 bytes representing the raw IEEE 754 bfloat16 bit pattern.
  • Endianness must match existing float32/float64 behavior.
  • NaN/Inf/±0/subnormal must round-trip correctly at the bit level (document NaN policy if canonicalized).

3) JavaScript Runtime (core requirement)

Provide a public strong type and conversions, not raw uint16 in public APIs.

3.1 Type definition
  • Type: BFloat16
  • Suggested representation: class BFloat16 { constructor(bits) { /* uint16 */ } }
  • Provide to_bits() / from_bits() equivalents.
3.2 Conversions (IEEE 754 compliant)
  • Convert float32 -> bfloat16 with round-to-nearest, ties-to-even.
  • Convert bfloat16 -> float32 exactly.
  • Correct handling of NaN/Inf/±0/subnormals.
3.3 Arrays / bfloat16_array
  • Provide BFloat16Array as a packed contiguous array of u16 bits.
  • Suggested storage: BFloat16Array backed by Uint16Array (packed u16 bits)
  • Mapping: bfloat16_array -> Uint16Array (or BFloat16Array wrapper), repeated bfloat16 -> BFloat16[] with fast-path for Uint16Array
3.4 Serialization boundary
  • Serializer/deserializer should read/write the underlying 16-bit pattern.
  • Public APIs should not expose raw uint16 for values.

4) Codegen requirement (JavaScript)

  • Generated fields for bfloat16 must use BFloat16.
  • Generated fields for bfloat16_array must use BFloat16Array or the packed container noted above.

5) Compiler/Reflection integration

  • Update type resolver/reflection to treat BFloat16 as the bfloat16 primitive (distinct from uint16).

6) Tests

  • Scalar conversions (±0, ±Inf, NaN, subnormals, max/min normal).
  • Array serialization tests for packed bfloat16_array.
  • Round-trip tests inside messages (including repeated/map use).

Describe alternatives you've considered

  1. Store BF16 as float32 in memory and downcast only on serialization.
  • Rejected: increases memory footprint and defers rounding, causing cross-language inconsistencies.
  1. Expose raw uint16 in public APIs.
  • Rejected: error-prone and inconsistent with strong-typed primitives.

Additional context

This issue mirrors the structure of the float16 work but targets bfloat16 + bfloat16_array.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions