Skip to content

feat(go): support [N]uint types array serializers#3201

Merged
chaokunyang merged 12 commits intoapache:mainfrom
ayush00git:feat/go-arrSerilization
Jan 27, 2026
Merged

feat(go): support [N]uint types array serializers#3201
chaokunyang merged 12 commits intoapache:mainfrom
ayush00git:feat/go-arrSerilization

Conversation

@ayush00git
Copy link
Contributor

@ayush00git ayush00git commented Jan 25, 2026

Referenced issue - #3015

Go Array Serialization Support

I have implemented optimized array serialization for uint16, uint32, and uint64 types in Go. These types now map to standard Fory type IDs (UINT16_ARRAY, UINT32_ARRAY, UINT64_ARRAY) instead of falling back to generic list serialization, ensuring better performance and cross-language compatibility.

Changes

  • Serialization Logic -

I updated array_primitive.go to include dedicated serializers:

  1. uint16ArraySerializer: Maps to UINT16_ARRAY
  2. uint32ArraySerializer: Maps to UINT32_ARRAY
  3. uint64ArraySerializer: Maps to UINT64_ARRAY

These serializers support both fast-path (direct memory copy on little-endian systems) and standard path serialization.

  • Registered the typeIds and serializers to the type_resolver.go

case UINT8:
		return &TypeInfo{Type: uint8Type, TypeID: typeID, Serializer: r.typeToSerializers[uint8Type], DispatchId: PrimitiveInt8DispatchId}
	case UINT16:
		return &TypeInfo{Type: uint16Type, TypeID: typeID, Serializer: r.typeToSerializers[uint16Type], DispatchId: PrimitiveInt16DispatchId}
	case UINT32, VAR_UINT32:
		return &TypeInfo{Type: uint32Type, TypeID: typeID, Serializer: r.typeToSerializers[uint32Type], DispatchId: PrimitiveInt32DispatchId}
	case UINT64, VAR_UINT64, TAGGED_UINT64:
		return &TypeInfo{Type: uint64Type, TypeID: typeID, Serializer: r.typeToSerializers[uint64Type], DispatchId: PrimitiveInt64DispatchId}
  • Updated slice.go - readSliceRefAndType to return the Type ID (uint32) it reads from the buffer. Previously, it only returned a boolean flag.

Before

func readSliceRefAndType(ctx *ReadContext, refMode RefMode, readType bool, value reflect.Value) (bool)

After

func readSliceRefAndType(ctx *ReadContext, refMode RefMode, readType bool, value reflect.Value) (bool, uint32)
  • Updated slice_primitive.go - all primitive slice serializers (like int64SliceSerializer, float32SliceSerializer, etc.).

Added a validation check: verifying that the returned typeId matches the expected wire type for that slice (e.g., []int64 now explicitly checks for INT64_ARRAY).
If the ID doesn't match, it now returns a "slice type mismatch" error instead of silently proceeding with potentially corrupt data.

	done, typeId := readSliceRefAndType(ctx, refMode, readType, value)
	if done || ctx.HasError() {
		return
	}
	if readType && typeId != uint32(BINARY) {
		ctx.SetError(DeserializationErrorf("slice type mismatch: expected BINARY (%d), got %d", BINARY, typeId))
		return
	}
  • Updated slice_dyn.go - Updated the generic sliceDynSerializer to validate that the incoming wire type is LIST (since dynamic slices always use the LIST protocol).

	done, typeId := readSliceRefAndType(ctx, refMode, readType, value)
	if done || ctx.HasError() {
		return
	}
	if readType && typeId != uint32(LIST) {
		ctx.SetError(DeserializationErrorf("slice type mismatch: expected LIST (%d), got %d", LIST, typeId))
		return
	}

Related issues

Closes #3015

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change?
  • Does this PR introduce any binary protocol compatibility change?

Benchmark

N/A

@ayush00git
Copy link
Contributor Author

Hey @chaokunyang
This is how I'm planning my execution for this PR.

  • Modify array_primitive.go to use UINT16_ARRAY, UINT32_ARRAY, UINT64_ARRAY for uint arrays, instead of previously used INT16_ARRAY, INT32_ARRAY, INT64_ARRAY.
  • Update type_resolver.go to register these correct type IDs.
  • Add float16 array support (using uint16 as storage) as it's not a native type in go.
  • Verify everything with tests.

@chaokunyang
Copy link
Collaborator

chaokunyang commented Jan 26, 2026

Hi @ayush00git , here is some of my comments:

  1. Add float16 array support (using uint16 as storage) as it's not a native type in go.: we can skip this currently.
  2. Could you add tests to test interoperability between array and slice? For example, serialize using arrya, then deserialzie into slcie, and serialize using slice, deserialize into array

@ayush00git
Copy link
Contributor Author

Hi @ayush00git , here is some of my comments:

  1. Add float16 array support (using uint16 as storage) as it's not a native type in go.: we can skip this currently.
  2. Could you add tests to test interoperability between array and slice? For example, serialize using arrya, then deserialzie into slcie, and serialize using slice, deserialize into array

Okk, i'll remove the float16 type, and for tests, i'm currently working on them and i'll keep them around your requests.

@ayush00git
Copy link
Contributor Author

Hey @chaokunyang
I'm done with the implementations, please give it a review..

Copy link
Collaborator

@chaokunyang chaokunyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great

@chaokunyang chaokunyang merged commit e76f878 into apache:main Jan 27, 2026
55 checks passed
@ayush00git ayush00git deleted the feat/go-arrSerilization branch January 27, 2026 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Go] Support array type serialization

2 participants