Skip to content

fix(javascript): align TypeMeta preamble constants with python/java/rust/go xlang bindings #3603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chaokunyang merged 1 commit into from
Apr 22, 2026
Merged

fix(javascript): align TypeMeta preamble constants with python/java/rust/go xlang bindings #3603

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 31 additions & 7 deletions javascript/packages/core/lib/meta/TypeMeta.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,18 @@ const pkgDecoder = new MetaStringDecoder(".", "_");
const typeNameEncoder = new MetaStringEncoder("$", ".");
const typeNameDecoder = new MetaStringDecoder("$", ".");

// Constants from Java implementation
const COMPRESS_META_FLAG = 1n << 63n;
const HAS_FIELDS_META_FLAG = 1n << 62n;
const META_SIZE_MASKS = 0xFF; // 22 bits
const NUM_HASH_BITS = 41;
// Constants shared with python/java/rust/go 0.17+. See e.g.
// python/pyfory/meta/typedef.py, java/.../TypeDef.java,
// rust/fory-core/src/meta/type_meta.rs, go/fory/type_def.go. The
// JavaScript binding previously placed COMPRESS_META_FLAG at bit 63
// and HAS_FIELDS_META_FLAG at bit 62, and used NUM_HASH_BITS = 41,
// producing an 8-byte TypeMeta preamble that no other xlang binding
// could decode. Aligning with the constants every other binding uses
// so NAMED_COMPATIBLE_STRUCT output is byte-compatible cross-binding.
const COMPRESS_META_FLAG = 1n << 9n;
const HAS_FIELDS_META_FLAG = 1n << 8n;
const META_SIZE_MASKS = 0xFF;
const NUM_HASH_BITS = 50;
const BIG_NAME_THRESHOLD = 0b111111;

const PRIMITIVE_TYPE_IDS = [
Expand Down Expand Up @@ -645,9 +652,26 @@ export class TypeMeta {
private prependHeader(buffer: Uint8Array, isCompressed: boolean, hasFieldsMeta: boolean): Uint8Array {
const metaSize = buffer.length;
const hash = x64hash128(buffer, 47);
let header = BigInt(hash.getUint32(0, false)) << 32n | BigInt(hash.getUint32(4, false));
// Read the high 64 bits of the 128-bit MurmurHash3 as a SIGNED
// int64 to match pyfory (`hash_buffer()[0]` unpacks `int64_t[0]`),
// java (`murmurhash3_x64_128(...)[0]` returns `long`), and rust
// (`.0 as i64`). Reading the same bytes as unsigned via two
// uint32 halves produces a different value after
// `<< (64 - NUM_HASH_BITS); abs()` whenever the hash's high bit
// is set -- unsigned BigInt can't go negative, so its sign-check
// is always false and the abs is a no-op. Signed int64 here
// matches the canonical behaviour of the other xlang bindings.
let header = hash.getBigInt64(0, false);
header = header << BigInt(64 - NUM_HASH_BITS);
header = header >= 0n ? header : -header; // Math.abs for bigint
// Arbitrary-precision abs + mask to 63 bits, matching pyfory's
// `abs(hash) & 0x7FFFFFFFFFFFFFFF`. The mask clears the sign bit
// so the COMPRESS_META_FLAG (bit 9) / HAS_FIELDS_META_FLAG
// (bit 8) / metaSize (low 8 bits) ORs below don't collide with
// residual hash bits.
if (header < 0n) {
header = -header;
}
header = header & 0x7FFFFFFFFFFFFFFFn;

if (isCompressed) {
header |= COMPRESS_META_FLAG;
Expand Down
Loading