v0.9.0

gRPC: parse in place, serialize without a new array

The generated gRPC marshaller no longer copies every inbound message through InputStream.readAllBytes(), and no longer allocates a new array for every outbound message. These came out of a performance review of the Oxia Java client (oxia-db/oxia-client-java#400, oxia-db/oxia-client-java#401, oxia-db/oxia-client-java#402), which picks them up with a LightProto upgrade and no code change.

Parse gRPC messages from the transport buffer instead of readAllBytes() (#23)

Over grpc-netty, parse() receives gRPC's BufferInputStream, which the existing zero-copy branch did not recognize, so every message was read into fresh 16 KiB chunks and copied again. A message that arrives in one transport buffer is now parsed in place and materialized before the buffer goes back to the transport; other KnownLength streams are read once into an exact-size array. A 100-byte message: 520 → 77 ns and 17,104 → 880 bytes allocated; 64 KiB: 6.6 → 2.2 µs (JDK 21). Generated stubs now need grpc-api 1.39 or later.

Parse gRPC messages spanning several transport buffers in place (#31)

Messages of 16 KiB or more span several HTTP/2 DATA frames, so #23 still copied them. They are now parsed in place as well: the marshaller marks the stream, which makes gRPC keep every buffer until close(), and reads across the frames through LightProtoCodec.SegmentedByteBuf. A 64 KiB value read as byte[]: 5.6 → 3.0 µs, 132 → 67 KB allocated (JDK 21). Smaller messages that straddle two frames are still copied, which is faster at that size.

Reuse a per-thread array to serialize outbound gRPC messages (#27)

stream() serializes into a per-thread array, and the stream hands it back when gRPC closes it, so steady-state sends allocate no array. Arrays above 4 MiB, gRPC's default maximum message size, are not kept. A 128 KiB message: 9.6 → 6.2 µs and 131 KB → 55 bytes allocated per message on one thread; 30.8 → 12.6 µs with 8 sending threads.

Return the wrapped array from bytes getters instead of copying it (#26)

The byte[] getters (getX(), getXAt(), and map getX(key) and forEachX()) return the message's own array when the value is held in exactly one whole array, as after materialize() or a byte[] setter, instead of copying it. With #23 and #31, a value received over gRPC is copied once. Behavior change: the returned array is shared with the message, so writing into it changes the field.

Writing back a parsed message

parseFrom() cached the number of bytes it consumed as the message's serialized size, and writeTo() sized its output from it. That only holds when LightProto writes the message back exactly as it arrived. For valid input that it doesn't (unknown fields or enum values, non-canonical varints, a packed field sent unpacked or the reverse, repeated singular fields, and more), writing back a parsed message, as a proxy or a gRPC service forwarding a request does, produced output with trailing bytes that are not part of the message, or wrote past the space it had reserved: in a pooled heap buffer, into the next buffer of the same chunk. A series of per-case fixes led to the general one in #43. A writer that wrote more than it reserved even on freshly built messages, for large sint32 values, is fixed in #42.

Don't size writeTo() output from the parsed wire size (#43)

getSerializedSize() now only trusts sizes it computed itself, which it caches with the sign bit set. A parsed message computes its size from its fields the first time it is written, as a built one does. parseFrom() is unchanged, and the clear() release gate (#19) still uses the wire size as its estimate of the retained data. The cost is that one size computation, the first time a parsed message is written back. NonCanonicalInputTest covers 32 such inputs; 23 failed before.

Per-case fixes

• Map entries with an omitted key or value now hold protobuf's defaults. String and bytes entries threw NullPointerException, and enum values read as null (#24).
• Map entries with an unknown enum value are dropped, as protobuf-java does, instead of being stored as null (#39).
• Unknown enum values (#25), unknown fields inside nested messages and map entries (#28), and map entries that repeat their key or value (#30) no longer leave the wire size cached.
• Generated per-field clearX() methods reset the cached size. Since at least 0.7.3, clearing a field after serializing or parsing kept the old size (#32).
• Bools are read as protobuf reads them: any non-zero varint is true. LightProto compared a 32-bit read with 1, so 2, 255 or 2^32 read as false (#35).

Write sint32 values as varints of at most 5 bytes (#42)

sint32 values with |v| ≥ 2^30 were sized as 5-byte varints but written as 10 bytes, on freshly built messages: toByteArray() threw ArrayIndexOutOfBoundsException, and writeTo() into a pooled heap buffer wrote 5 bytes into the next buffer.

Write uint32 values as varints of at most 5 bytes (#44)

uint32 values ≥ 2^31 were written as the 10-byte sign extension. They are now written in 5 bytes, as protobuf does. This changes LightProto's output bytes for those values; every reader decodes both encodings to the same value.

Grow composite write targets before choosing the write path (#40)

writeTo() into a CompositeByteBuf that had to grow, such as a full single-component composite or any empty compositeDirectBuffer(), threw UnsupportedOperationException.

Performance vs v0.8.2

JDK 26, Apple M1 Max, six interleaved A/B rounds (one fork per version per round), pooled direct buffers as targets. Time per operation, lower is better; Δ is the median of the per-round ratios.

	v0.8.2	v0.9.0	Δ
Serialize (small)
Pulsar BaseCommand (~50 B)	22.5 ns	24.6 ns	+8.6%
Pulsar MessageMetadata (~70 B)	68.3 ns	68.0 ns	-0.2%
AddressBook	40.6 ns	40.5 ns	-0.2%
AddressBook fill + serialize	66.1 ns	64.9 ns	-1.9%
Frame (tiny)	6.3 ns ¹	6.6 ns	+5%
Deserialize
Pulsar BaseCommand	22.1 ns	22.0 ns	-0.5%
Pulsar MessageMetadata	41.7 ns	42.4 ns	+1.5%
AddressBook	37.0 ns	38.0 ns	+2.6%
Frame (tiny)	10.3 ns	10.4 ns	+1.1%
Frame + read strings	19.2 ns	19.1 ns	-0.6%
Serialize (large, to a direct buffer)
topic list 600 B	26.5 ns	27.0 ns	+0.7%
topic list 1 KB	39.3 ns	39.1 ns	-0.5%
topic list 3 KB	92.8 ns	92.6 ns	-0.3%
varint-dense 2 KB	0.73 µs	0.73 µs	+0.1%
topic list 16 KB	0.40 µs	0.40 µs	-0.2%
topic list 100 KB	3.65 µs	3.65 µs	-1.1%
bytes payload 2 MB	50 µs	48 µs	-2.5%
topic list 4.6 MB	166 µs	166 µs	-0.9%

¹ v0.8.2's Frame serialize ran at 6.3 ns in 2 of the 6 forks and at 28.8 ns, a 4.5× slower JIT mode, in the other 4; the row compares against the fast mode. v0.9.0 ran at 6.6 ns in all 6.

Most rows are within 3% of v0.8.2. The correctness fixes add a compare or two to clear() and to nested parses (#28, #43), and #45 wins part of that back on the write side. Pulsar BaseCommand serialize is the exception, at +8.6% (about 2 ns). #43 cost it about 13%, all in the compiled BaseCommand._writeTo(), and #45 recovers about half of that. What remains is in how C2 compiles the varint writes of the nested CommandSend there, code this release does not change.

Write nested length prefixes from the cached size (#45)

_writeTo() wrote each nested message's length prefix with getSerializedSize(), so C2 compiled the nested message's whole size walk into the parent's _writeTo(). It never runs there: every write computes the sizes first. After #43, Pulsar BaseCommand serialize was about 13% slower than in 0.8.2, all of it in that compiled _writeTo(). _writeTo() now reads the size through an internal _sizeForWrite(), which returns the cached size and falls back to getSerializedSize(), so the size walk is no longer compiled in. That recovers about half of the gap, and makes AddressBook fill + serialize 5% faster than before the change.

Write JSON strings in runs instead of one character at a time (#33)

toJson() and writeJsonTo() encoded every character of a string field as its own String: a 48-character topic name took 735 ns, now 53 ns. Characters outside the Basic Multilingual Plane, such as emoji, were written as ?? and are now encoded correctly.

Compute the payload size of packed varint fields once per serialization (#41)

Packed varint fields, the proto3 default for repeated integers and enums, were sized twice per serialization. writeTo() of a packed int64 field is 14-19% faster.

Copy heap-buffer strings with System.arraycopy instead of Unsafe (#38)

Strings read from heap buffers, as after parseFrom(byte[]), no longer pay the per-call check that JDK 24+ adds to sun.misc.Unsafe memory access: 2-3% on parses that read strings.

Upgrade notes

• byte[] getters can return the message's own array, so writing into it changes the field (#26).
• uint32 values ≥ 2^31 are now written in 5 bytes instead of 10 (#44).
• Generated gRPC stubs need grpc-api 1.39 or later (#23).

Also included

• Text format: a map entry without a value is kept with the default value, and a key repeated in a later entry replaces its value instead of merging into it, as in protobuf-java (#36).
• The unused ByteBuf writers are removed from the generated LightProtoCodec (#34).
• Tests build against gRPC 1.75.0 (#29).