diff --git a/.agents/languages/java.md b/.agents/languages/java.md index caf4085a3b..bc58401a82 100644 --- a/.agents/languages/java.md +++ b/.agents/languages/java.md @@ -6,6 +6,12 @@ Load this file when changing anything under `java/` or when Java drives a cross- - Run all Maven commands from within `java/`. - Changes under `java/` must pass code style checks and tests. +- When changes are limited to `fory-json` or `fory-format`, do not run `fory-core` + tests. Install the changed module and its reactor dependencies with + `-am install -DskipTests`, then run `test` with only the changed module selected + and without `-am`. In particular, never use `-pl fory-json -am test` or + `-pl fory-format -am test`, because Maven propagates the test phase to + `fory-core`. - If tests already passed and the only later change is Maven Spotless formatting, do not rerun tests solely because of that formatting pass. Verify formatting with `spotless:check` and inspect the diff/status instead. diff --git a/docs/guide/java/json-support.md b/docs/guide/java/json-support.md index 518158eed9..3aa5a0e7d1 100644 --- a/docs/guide/java/json-support.md +++ b/docs/guide/java/json-support.md @@ -177,8 +177,11 @@ ForyJson json = .build(); ``` -`withConcurrencyLevel` controls reusable operation states, not a caller limit. Extra concurrent -operations use temporary state rather than one global lock. +`withConcurrencyLevel` sets the maximum number of root operations that execute concurrently. +Additional callers wait until one of those fixed execution states is available. Root APIs on one +`ForyJson` instance are not reentrant: a custom codec must continue through the concrete reader or +writer passed to it instead of calling `toJson`, `toJsonBytes`, `writeJsonTo`, or `fromJson` on that +instance. ## Object mapping @@ -286,7 +289,7 @@ are rejected. | `withClassLoader` | Snapshotted context loader, then Fory loader | Resolve annotation subtype class names | | `maxDepth` | `20` | Maximum nested object/array depth | | `withMaxCachedFieldNames` | `DEFAULT_MAX_CACHED_FIELD_NAMES` (`8192`) | Field-name cache entries per reader; zero disables it | -| `withConcurrencyLevel` | `max(1, 2 * processors)` | Reusable operation-state count | +| `withConcurrencyLevel` | `max(1, 2 * processors)` | Maximum concurrent root operations | | `withBufferSizeLimitBytes` | 2 MiB | Reusable capacity retained by each pooled writer | | `registerCodec` | None | Exact-class complete-value codec | | `registerMixin` | None | Annotation Mixin for its exact declared target | @@ -682,12 +685,12 @@ The complete group occupies one position in parent serialization order. Position is preserved inside the group. Input matches parent fixed properties first, flattened properties second, and dynamic Any members last. -Fory rejects final-name or name-hash collisions, recursive chains made only of unwrapped -properties, parameterized children, JSON Any children, polymorphic or custom-codec child roots, -and scalar, array, collection, or Map children. Flatten Maps with `JsonAnyProperty`, -`JsonAnyGetter`, or `JsonAnySetter`. An unwrapped property cannot use `JsonProperty.value`, a -non-default `JsonProperty.include`, or `JsonCodec`; ordinary leaf properties inside the child keep -their normal annotations. +Fory rejects duplicate final names, recursive chains made only of unwrapped properties, +parameterized children, JSON Any children, polymorphic or custom-codec child roots, and scalar, +array, collection, or Map children. Flatten Maps with `JsonAnyProperty`, `JsonAnyGetter`, or +`JsonAnySetter`. An unwrapped property cannot use `JsonProperty.value`, a non-default +`JsonProperty.include`, or `JsonCodec`; ordinary leaf properties inside the child keep their normal +annotations. ### Dynamic object members @@ -760,15 +763,11 @@ Dynamic keys are emitted unchanged in Map iteration order. A null Map emits noth Map value emits JSON null regardless of fixed-property null settings. Null and non-String output keys are rejected. Raw Maps, wildcard or unresolved keys, and non-String key types are invalid. Declared fixed members, including members excluded from reading, are not delivered to an Any -input. Output keys whose Fory field-name hash conflicts with a fixed property are rejected, -including differently spelled hash collisions. Fory does not inspect an Any Map for a key whose -name or Fory field-name hash conflicts with an inline subtype discriminator. An exact-name output -key emits a duplicate JSON member; on input, a differently spelled hash collision is classified as -the discriminator by the child field table. Applications must keep dynamic keys distinct from the -active discriminator by both name and hash. Repeated unknown names replace the Map value; an -any-setter is called for every occurrence. Fixed input lookup is also hash-based, so a differently -spelled colliding name follows the fixed member instead of Any handling. Escaped input names are -decoded before delivery. +input. An output key that conflicts with a fixed property is rejected. Fory does not inspect an Any +Map for a key that duplicates an inline subtype discriminator; such a key emits a duplicate JSON +member. Applications must keep dynamic keys distinct from the active discriminator. Repeated +unknown names replace the Map value; an any-setter is called for every occurrence. Escaped input +names are decoded before delivery. ### `JsonCreator` diff --git a/java/fory-json/README.md b/java/fory-json/README.md index bbda440c32..047c7a8e02 100644 --- a/java/fory-json/README.md +++ b/java/fory-json/README.md @@ -196,9 +196,11 @@ ForyJson json = .build(); ``` -`withConcurrencyLevel` configures the number of reusable operation states, not a maximum number of -concurrent callers. When all reusable states are busy, Fory JSON creates a temporary state rather -than serializing callers through one global lock. +`withConcurrencyLevel` sets the maximum number of root operations that execute concurrently. +Additional callers wait until one of those fixed execution states is available. Root APIs on one +`ForyJson` instance are not reentrant: a custom codec must continue through the concrete reader or +writer passed to it instead of calling `toJson`, `toJsonBytes`, `writeJsonTo`, or `fromJson` on that +instance. ## Java object mapping @@ -360,7 +362,7 @@ original key type. Null map keys are rejected. | `withClassLoader(loader)` | Snapshotted thread context loader, then Fory JSON loader | Resolve annotation-declared subtype class names | | `maxDepth(int)` | `20` | Maximum nested object/array depth for reads and writes | | `withMaxCachedFieldNames(int)` | `DEFAULT_MAX_CACHED_FIELD_NAMES` (`8192`) | Field-name cache entries per reader; zero disables caching | -| `withConcurrencyLevel(int)` | `max(1, 2 * processors)` | Number of reusable concurrent operation states | +| `withConcurrencyLevel(int)` | `max(1, 2 * processors)` | Maximum concurrent root operations | | `withBufferSizeLimitBytes(int)` | 2 MiB | Maximum reusable capacity retained by each pooled writer | | `registerCodec(type, codec)` | None | Replace the exact class's complete JSON codec | | `registerMixin(mixinType)` | None | Apply one annotation Mixin to its exact declared target | @@ -754,12 +756,11 @@ position it, and `JsonPropertyOrder` selects it by Java logical property name. T property order remains intact. Parent fields are matched before flattened fields, which are matched before dynamic Any handling. -Fory rejects duplicate or hash-colliding final names, recursive chains made only of unwrapped -properties, parameterized children, JSON Any children, polymorphic or custom-codec child roots, -and scalar, array, collection, or Map children. Use `JsonAnyProperty`, `JsonAnyGetter`, or -`JsonAnySetter` to flatten a Map. `JsonProperty.value`, non-default `JsonProperty.include`, and -`JsonCodec` are not valid on an unwrapped property; ordinary child leaf properties may still use -them. +Fory rejects duplicate final names, recursive chains made only of unwrapped properties, +parameterized children, JSON Any children, polymorphic or custom-codec child roots, and scalar, +array, collection, or Map children. Use `JsonAnyProperty`, `JsonAnyGetter`, or `JsonAnySetter` to +flatten a Map. `JsonProperty.value`, non-default `JsonProperty.include`, and `JsonCodec` are not +valid on an unwrapped property; ordinary child leaf properties may still use them. ### Dynamic object members @@ -854,15 +855,11 @@ Dynamic keys are exact JSON member names and retain Map iteration order. A null members, and a null Map value writes JSON null regardless of fixed-property null settings. Null and non-String output keys are rejected. Raw Maps, wildcard or unresolved keys, and non-String key types are invalid. Declared fixed members, including members excluded from reading, are not -delivered to an Any input. An output key is rejected when its Fory field-name hash conflicts with a -fixed property; this also covers differently spelled hash collisions. Fory does not inspect an Any -Map for a key whose name or Fory field-name hash conflicts with an inline subtype discriminator. An -exact-name output key writes a duplicate JSON member; on input, a differently spelled hash -collision is classified as the discriminator by the child field table. Applications must keep -dynamic keys distinct from the active discriminator by both name and hash. Fixed input lookup is -also hash-based, so a differently spelled colliding name follows the fixed member instead of Any -handling. Repeated unknown input names replace the prior Map value, while an any-setter is invoked -for every occurrence. Escaped input names are decoded before delivery. +delivered to an Any input. An output key that conflicts with a fixed property is rejected. Fory +does not inspect an Any Map for a key that duplicates an inline subtype discriminator; such a key +writes a duplicate JSON member. Applications must keep dynamic keys distinct from the active +discriminator. Repeated unknown input names replace the prior Map value, while an any-setter is +invoked for every occurrence. Escaped input names are decoded before delivery. ### `JsonCreator` diff --git a/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java b/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java index 6fc3d0d96c..d4841a6379 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/ForyJson.java @@ -27,6 +27,7 @@ import java.lang.reflect.WildcardType; import java.util.Objects; import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.locks.LockSupport; import org.apache.fory.json.reader.Latin1JsonReader; import org.apache.fory.json.reader.Utf16JsonReader; import org.apache.fory.json.reader.Utf8JsonReader; @@ -61,13 +62,16 @@ * path selection is observable by custom codecs and is therefore not interchangeable even when a * Latin1 string contains only ASCII. * - *

The facade has no close lifecycle. Contended operations borrow another pooled state or create - * a temporary unpooled state instead of serializing all callers through one root lock. Java {@code - * null} writes as JSON {@code null}; JSON {@code null} returns {@code null} for reference targets - * and is rejected for primitive root targets. + *

The facade has no close lifecycle. It owns exactly the configured number of execution states; + * a root operation waits when every state is in use. Root APIs on one instance are not reentrant. A + * custom codec must continue through the concrete reader or writer passed to it instead of invoking + * another root API on the same instance. Java {@code null} writes as JSON {@code null}; JSON {@code + * null} returns {@code null} for reference targets and is rejected for primitive root targets. */ public final class ForyJson { private static final int HOME_SLOT_RETRIES = 2; + private static final int CONTENDED_YIELD_SCANS = 32; + private static final long CONTENDED_PARK_NANOS = 100L; private static final int INITIAL_BUFFER_SIZE = 8192; private static final int RETAINED_UTF16_BYTES = 64 * 1024; private static final byte[] EMPTY_BYTES = new byte[0]; @@ -78,8 +82,6 @@ public final class ForyJson { /** Default maximum number of short, unescaped ASCII field names cached by each JSON reader. */ public static final int DEFAULT_MAX_CACHED_FIELD_NAMES = 8192; - private final JsonConfig config; - private final JsonSharedRegistry sharedRegistry; private final int homeSlotMask; private final PooledState[] slots; @@ -88,13 +90,14 @@ public final class ForyJson { } ForyJson(JsonConfig config, JsonSharedRegistry sharedRegistry) { - this.config = config; - this.sharedRegistry = sharedRegistry; int poolSize = config.concurrencyLevel(); homeSlotMask = Integer.highestOneBit(poolSize) - 1; + // This fixed array is the only JsonState owner. Each state's three readers own their configured + // field-name cache limits, so creating execution states outside this array would make the + // number of caches unbounded. slots = new PooledState[poolSize]; for (int i = 0; i < poolSize; i++) { - slots[i] = new PooledState(new JsonState(config, sharedRegistry), true); + slots[i] = new PooledState(new JsonState(config, sharedRegistry)); } } @@ -103,7 +106,13 @@ public static ForyJsonBuilder builder() { return new ForyJsonBuilder(); } - /** Serializes {@code value} as one complete JSON document backed by a detached String. */ + /** + * Serializes {@code value} as one complete JSON document backed by a detached String. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link StringJsonWriter} passed to its {@code + * writeString} method instead of invoking a {@code ForyJson} root API. + */ public String toJson(Object value) { PooledState entry = acquire(); JsonState state = entry.state; @@ -136,6 +145,10 @@ public String toJson(Object value) { *

This overload is required when the declared type owns a closed {@code JsonSubTypes} table. A * non-null value must be assignable to the declared type. Primitive declarations accept only * their exact boxed carrier and reject null; {@code void} is never a JSON value type. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link StringJsonWriter} passed to its {@code + * writeString} method instead of invoking a {@code ForyJson} root API. */ public String toJson(T value, Class declaredType) { requireDeclaredType(declaredType); @@ -148,6 +161,10 @@ public String toJson(T value, Class declaredType) { * *

An explicit declared type controls the complete root schema, including closed subtype * metadata inside generic containers. A non-null value must be assignable to its raw type. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link StringJsonWriter} passed to its {@code + * writeString} method instead of invoking a {@code ForyJson} root API. */ public String toJson(T value, TypeRef declaredType) { requireDeclaredType(declaredType); @@ -157,7 +174,13 @@ public String toJson(T value, TypeRef declaredType) { return toJsonDeclared(value, declaredType.getType(), rawType); } - /** Serializes {@code value} as one complete JSON document in a detached UTF-8 byte array. */ + /** + * Serializes {@code value} as one complete JSON document in a detached UTF-8 byte array. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link Utf8JsonWriter} passed to its {@code + * writeUtf8} method instead of invoking a {@code ForyJson} root API. + */ public byte[] toJsonBytes(Object value) { PooledState entry = acquire(); JsonState state = entry.state; @@ -184,14 +207,26 @@ public byte[] toJsonBytes(Object value) { } } - /** Serializes {@code value} as UTF-8 using {@code declaredType}'s codec. */ + /** + * Serializes {@code value} as UTF-8 using {@code declaredType}'s codec. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link Utf8JsonWriter} passed to its {@code + * writeUtf8} method instead of invoking a {@code ForyJson} root API. + */ public byte[] toJsonBytes(T value, Class declaredType) { requireDeclaredType(declaredType); validateWriteValue(value, declaredType); return toJsonBytesDeclared(value, declaredType, declaredType); } - /** Serializes {@code value} as UTF-8 using the generic codec captured by {@code declaredType}. */ + /** + * Serializes {@code value} as UTF-8 using the generic codec captured by {@code declaredType}. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link Utf8JsonWriter} passed to its {@code + * writeUtf8} method instead of invoking a {@code ForyJson} root API. + */ public byte[] toJsonBytes(T value, TypeRef declaredType) { requireDeclaredType(declaredType); validateDeclaredType(declaredType.getType()); @@ -205,6 +240,10 @@ public byte[] toJsonBytes(T value, TypeRef declaredType) { * *

The complete document is buffered before one write to the stream. This method neither * flushes nor closes the caller-owned stream. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link Utf8JsonWriter} passed to its {@code + * writeUtf8} method instead of invoking a {@code ForyJson} root API. */ public void writeJsonTo(Object value, OutputStream output) { Objects.requireNonNull(output, "output"); @@ -237,6 +276,10 @@ public void writeJsonTo(Object value, OutputStream output) { /** * Writes UTF-8 JSON using {@code declaredType}'s codec without flushing or closing {@code * output}. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link Utf8JsonWriter} passed to its {@code + * writeUtf8} method instead of invoking a {@code ForyJson} root API. */ public void writeJsonTo(T value, Class declaredType, OutputStream output) { requireDeclaredType(declaredType); @@ -247,6 +290,10 @@ public void writeJsonTo(T value, Class declaredType, OutputStream output) /** * Writes UTF-8 JSON using the generic codec captured by {@code declaredType}, without flushing or * closing {@code output}. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must write nested content through the {@link Utf8JsonWriter} passed to its {@code + * writeUtf8} method instead of invoking a {@code ForyJson} root API. */ public void writeJsonTo(T value, TypeRef declaredType, OutputStream output) { requireDeclaredType(declaredType); @@ -394,6 +441,11 @@ private static void validateDeclaredType(Type type) { /** * Parses exactly one JSON value from {@code json} using {@code type} as its declared Java type. * Trailing non-whitespace content is rejected. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must consume nested content through the {@link Latin1JsonReader} or {@link + * Utf16JsonReader} passed to its representation-specific read method instead of invoking a {@code + * ForyJson} root API. */ public T fromJson(String json, Class type) { PooledState entry = acquire(); @@ -417,6 +469,11 @@ public T fromJson(String json, Class type) { /** * Parses exactly one JSON value using a generic type captured by {@link TypeRef}. Trailing * non-whitespace content is rejected. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must consume nested content through the {@link Latin1JsonReader} or {@link + * Utf16JsonReader} passed to its representation-specific read method instead of invoking a {@code + * ForyJson} root API. */ public T fromJson(String json, TypeRef typeRef) { PooledState entry = acquire(); @@ -441,6 +498,10 @@ public T fromJson(String json, TypeRef typeRef) { /** * Parses exactly one UTF-8 JSON value from {@code bytes} using {@code type} as its declared Java * type. Trailing non-whitespace content is rejected. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must consume nested content through the {@link Utf8JsonReader} passed to its {@code + * readUtf8} method instead of invoking a {@code ForyJson} root API. */ public T fromJson(byte[] bytes, Class type) { PooledState entry = acquire(); @@ -464,6 +525,10 @@ public T fromJson(byte[] bytes, Class type) { /** * Parses exactly one UTF-8 JSON value using a generic type captured by {@link TypeRef}. Trailing * non-whitespace content is rejected. + * + *

This root API is not reentrant on the same instance. A custom codec invoked by this + * operation must consume nested content through the {@link Utf8JsonReader} passed to its {@code + * readUtf8} method instead of invoking a {@code ForyJson} root API. */ public T fromJson(byte[] bytes, TypeRef typeRef) { PooledState entry = acquire(); @@ -490,7 +555,7 @@ private PooledState acquire() { PooledState[] slots = this.slots; if (slots.length == 1) { PooledState entry = slots[0]; - return entry.tryAcquire() ? entry : newOverflowState(); + return entry.tryAcquire() ? entry : acquireContended(0); } // A Thread's identity hash is stable for both platform and virtual threads, but retaining the // Thread is unnecessary. The hash selects only a cache-affine home state; the lease remains @@ -501,37 +566,45 @@ private PooledState acquire() { return entry.tryAcquire() ? entry : acquireContended(slotIndex); } - private PooledState newOverflowState() { - return new PooledState(new JsonState(config, sharedRegistry), false); - } - private void release(PooledState entry) { entry.release(); } private PooledState acquireContended(int slotIndex) { - PooledState entry; - for (int i = 1; i < HOME_SLOT_RETRIES; i++) { - entry = tryBorrowSlot(slotIndex); - if (entry != null) { - return entry; - } - } - int index = slotIndex + 1; - if (index == slots.length) { - index = 0; - } - for (int i = 1; i < slots.length; i++) { - entry = tryBorrowSlot(index); - if (entry != null) { - return entry; + int failedScans = 0; + while (true) { + PooledState entry; + for (int i = 1; i < HOME_SLOT_RETRIES; i++) { + entry = tryBorrowSlot(slotIndex); + if (entry != null) { + return entry; + } } - index++; + int index = slotIndex + 1; if (index == slots.length) { index = 0; } + for (int i = 1; i < slots.length; i++) { + entry = tryBorrowSlot(index); + if (entry != null) { + return entry; + } + index++; + if (index == slots.length) { + index = 0; + } + } + // Yield through brief contention, then park after repeated complete misses to prevent + // sustained saturation from consuming a CPU per waiter. parkNanos is imprecise and release + // does not notify it, so the delay stays tiny; waiter registration would add shared state + // and work to release and uncontended paths. + if (failedScans < CONTENDED_YIELD_SCANS) { + failedScans++; + Thread.yield(); + } else { + LockSupport.parkNanos(CONTENDED_PARK_NANOS); + } } - return newOverflowState(); } private PooledState tryBorrowSlot(int index) { @@ -612,13 +685,11 @@ private static ForyJsonException primitiveNull(Class type) { /** Permanently owns one execution state and leases it to at most one root operation. */ private static final class PooledState { private final JsonState state; - private final boolean pooled; private final AtomicInteger leased; - private PooledState(JsonState state, boolean pooled) { + private PooledState(JsonState state) { this.state = state; - this.pooled = pooled; - leased = new AtomicInteger(pooled ? 0 : 1); + leased = new AtomicInteger(); } private boolean tryAcquire() { @@ -626,11 +697,9 @@ private boolean tryAcquire() { } private void release() { - if (pooled) { - // A slot keeps its state reference for its whole lifetime. Publishing only the lease avoids - // a reference-store GC barrier and still makes all state cleanup visible to the next owner. - leased.lazySet(0); - } + // A slot keeps its state reference for its whole lifetime. Publishing only the lease avoids + // a reference-store GC barrier and still makes all state cleanup visible to the next owner. + leased.lazySet(0); } } diff --git a/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java b/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java index ee4cfeec43..102b36b0a8 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/ForyJsonBuilder.java @@ -153,7 +153,11 @@ public ForyJsonBuilder withMaxCachedFieldNames(int maxCachedFieldNames) { return this; } - /** Sets the number of reusable execution states available to concurrent root operations. */ + /** + * Sets the maximum number of root operations that can execute concurrently. + * + *

Additional callers wait until an execution state becomes available. + */ public ForyJsonBuilder withConcurrencyLevel(int concurrencyLevel) { if (concurrencyLevel < 1) { throw new IllegalArgumentException("concurrencyLevel must be positive"); diff --git a/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java b/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java index fb665e0aa1..2d82636d67 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/JsonConfig.java @@ -37,11 +37,10 @@ * *

Scalar settings are fixed at construction. The codec registry is builder-owned mutable input * and is copied immediately by the runtime's shared registry; the JSON runtime never mutates it. - * The type checker uses identity semantics because checker instances may carry different user - * policy despite sharing a class. {@link #getCodegenHash()} identifies only settings that can - * change generated source; runtime-only settings such as depth and asynchronous scheduling do not - * fragment generated class names. Concurrency, per-reader field-name cache, and retained - * writer-buffer limits are also runtime-only and do not fragment generated class names. + * {@link #getCodegenHash()} identifies only settings that can change generated source; runtime-only + * settings such as depth and asynchronous scheduling do not fragment generated class names. + * Concurrency, per-reader field-name cache, and retained writer-buffer limits are also runtime-only + * and do not fragment generated class names. */ public final class JsonConfig { private static final int MAX_CACHED_FIELD_NAMES = 1 << 29; @@ -60,7 +59,6 @@ public final class JsonConfig { private final Map, Class> mixins; private final JsonTypeChecker typeChecker; private final JsonTypeCheckContext typeCheckContext; - private final String codecRegistryKey; private final CodegenKey codegenKey; private transient int codegenHash; @@ -94,7 +92,7 @@ public final class JsonConfig { this.mixins = immutableMixins(mixins); this.typeChecker = typeChecker; typeCheckContext = new JsonTypeCheckContext(); - codecRegistryKey = codecRegistry.codegenKey(); + String codecRegistryKey = codecRegistry.codegenKey(); codegenKey = new CodegenKey( writeNullFields, @@ -171,48 +169,6 @@ public JsonTypeCheckContext typeCheckContext() { return typeCheckContext; } - @Override - public boolean equals(Object other) { - if (this == other) { - return true; - } - if (other == null || getClass() != other.getClass()) { - return false; - } - JsonConfig that = (JsonConfig) other; - return writeNullFields == that.writeNullFields - && codegenEnabled == that.codegenEnabled - && asyncCompilationEnabled == that.asyncCompilationEnabled - && propertyDiscoveryEnabled == that.propertyDiscoveryEnabled - && propertyNamingStrategy == that.propertyNamingStrategy - && classLoader == that.classLoader - && maxDepth == that.maxDepth - && maxCachedFieldNames == that.maxCachedFieldNames - && concurrencyLevel == that.concurrencyLevel - && bufferSizeLimitBytes == that.bufferSizeLimitBytes - && typeChecker == that.typeChecker - && Objects.equals(codecRegistryKey, that.codecRegistryKey) - && mixins.equals(that.mixins); - } - - @Override - public int hashCode() { - int result = Boolean.hashCode(writeNullFields); - result = 31 * result + Boolean.hashCode(codegenEnabled); - result = 31 * result + Boolean.hashCode(asyncCompilationEnabled); - result = 31 * result + Boolean.hashCode(propertyDiscoveryEnabled); - result = 31 * result + propertyNamingStrategy.hashCode(); - result = 31 * result + System.identityHashCode(classLoader); - result = 31 * result + maxDepth; - result = 31 * result + maxCachedFieldNames; - result = 31 * result + concurrencyLevel; - result = 31 * result + bufferSizeLimitBytes; - result = 31 * result + System.identityHashCode(typeChecker); - result = 31 * result + codecRegistryKey.hashCode(); - result = 31 * result + mixins.hashCode(); - return result; - } - private static Map, Class> immutableMixins(Map, Class> registrations) { if (registrations.isEmpty()) { return Collections.emptyMap(); diff --git a/java/fory-json/src/main/java/org/apache/fory/json/annotation/JsonUnwrapped.java b/java/fory-json/src/main/java/org/apache/fory/json/annotation/JsonUnwrapped.java index 7850c9682d..24f4936155 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/annotation/JsonUnwrapped.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/annotation/JsonUnwrapped.java @@ -45,8 +45,8 @@ *

The complete group occupies one position in the containing object's serialization order and is * identified there by its Java logical property name. A creator-only parameter defines a read-only * group; its required {@link JsonProperty#value()} identifies the creator argument and is not a - * JSON wrapper name. Recursive chains made only of unwrapped properties and final-name or name-hash - * collisions are rejected when model metadata is resolved. + * JSON wrapper name. Recursive chains made only of unwrapped properties and duplicate final names + * are rejected when model metadata is resolved. */ @Documented @Retention(RetentionPolicy.RUNTIME) diff --git a/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonCreatorInfo.java b/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonCreatorInfo.java index 7e2a5cb110..81a9c2e5e5 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonCreatorInfo.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonCreatorInfo.java @@ -87,7 +87,7 @@ public Object[] newArguments() { public int index(long hash) { // Creator arity is deliberately finite and normally small. A linear exact-hash table avoids a - // second object graph and is allocation-free; construction rejects every hash collision. + // second object graph and is allocation-free. for (int i = 0; i < hashes.length; i++) { if (hashes[i] == hash) { return i; diff --git a/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldNameHash.java b/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldNameHash.java index b30c79edec..b9ac6a9418 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldNameHash.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldNameHash.java @@ -20,12 +20,18 @@ package org.apache.fory.json.meta; /** - * Incremental field-name hash shared by all concrete readers and metadata lookup tables. + * Internal field-name key shared by concrete readers and metadata lookup tables. * - *

Nonzero Latin1 names of at most eight bytes use their packed bytes as a collision-free fast - * value. All other names use the same FNV-style incremental hash while decoding. {@link - * JsonFieldTable} rejects metadata collisions up front, so a successful runtime hash lookup can - * return the canonical field without allocating or comparing a decoded String. + *

Non-empty names of at most eight nonzero Latin1 code units use their bytes packed from least + * to most significant. All other names use an FNV-1a-style hash over the decoded UTF-16 code units. + * Object metadata owners that dispatch only by this key reject duplicate keys before publishing + * their lookup tables, allowing readers to dispatch without materializing a String. An untrusted + * input collision grants no additional field or enum capability because the input could select the + * same target by sending its canonical JSON name. + * + *

For both inline-property and wrapper inclusions, closed-subtype lookup uses this key only to + * select a candidate. It still compares the complete decoded discriminator property and logical + * subtype name before selecting a class. */ public final class JsonFieldNameHash { public static final long MAGIC_HASH_CODE = 0xcbf29ce484222325L; diff --git a/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldTable.java b/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldTable.java index e5e36b8a60..3b71d1a6b4 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldTable.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/meta/JsonFieldTable.java @@ -28,9 +28,8 @@ *

The table is built at a low load factor during object metadata construction and stores both * the field object and its ordered read index. Any-enabled metadata may also store declared fixed * names that have no read sink in a separate hash table so those names are skipped rather than - * captured as dynamic members. Concrete readers probe by the hash computed while reading the member - * name, avoiding String materialization. Hash collisions between declared fields are rejected - * during construction because runtime lookup deliberately performs no secondary name comparison. + * captured as dynamic members. Concrete readers probe by the internal {@link JsonFieldNameHash} key + * computed while reading the member name, avoiding String materialization. */ public final class JsonFieldTable { @Internal public static final int UNKNOWN = -1; diff --git a/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java b/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java index 516c7c2682..590a72d5e4 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/reader/JsonReader.java @@ -190,6 +190,8 @@ public final int scanObjectStringField(JsonSubtypeScanInfo info) { throw errorAt("Expected ':'", cursor); } cursor = scanWhitespace(cursor + 1); + // The hash selects only the declared discriminator-property candidate. The complete + // decoded member name must match before this field can control subtype selection. if (fieldHash == info.propertyHash() && matchesScannedString(fieldStart, fieldEnd, info.property())) { if (found >= 0) { @@ -201,6 +203,8 @@ && matchesScannedString(fieldStart, fieldEnd, info.property())) { int valueStart = cursor; int valueEnd = scanStringEnd(valueStart); int candidate = info.nameIndex(scanStringHash(valueStart, valueEnd)); + // Logical subtype names use the same two-stage contract: select by hash, then compare the + // complete decoded name before returning the closed-table class index. if (candidate < 0 || !matchesScannedString(valueStart, valueEnd, info.name(candidate))) { throw errorAt("Unknown JSON subtype discriminator", valueStart); } @@ -243,7 +247,12 @@ && matchesScannedString(fieldStart, fieldEnd, info.property())) { protected abstract boolean matchesScannedString(int start, int end, String expected); - /** Reads one subtype name from a fixed validated table without materializing a String. */ + /** + * Reads one wrapper subtype name from a fixed validated table without materializing a String. + * + *

Concrete readers may use the decoded-name hash to select a candidate, but must compare the + * complete decoded name before returning its closed-table class index. + */ public abstract int readSubtypeName(JsonSubtypeScanInfo info); private final AsciiStringView asciiStringView = new AsciiStringView(this); diff --git a/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java b/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java index 285bf4247b..2fe6f7f735 100644 --- a/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java +++ b/java/fory-json/src/main/java/org/apache/fory/json/resolver/JsonSharedRegistry.java @@ -146,9 +146,8 @@ *

Accepted type-check results are cached by class name up to a bounded 8192-entry shared cache. * Once full, new names are checked on every resolution rather than growing attacker-controlled * state. Common short field names admitted by reader-local caches are published here for - * best-effort String reference reuse across readers. Reader-local admission is the only field-name - * capacity gate; the shared field-name map has no explicit limit. Source-generated model companions - * and JIT-generated class futures are shared here; concrete JIT codec instances, ordinary type + * best-effort String reference reuse across readers. Source-generated model companions and + * JIT-generated class futures are shared here; concrete JIT codec instances, ordinary type * bindings, graph construction, JIT locks, and publication remain resolver-local. A fresh {@link * JsonJITContext} is therefore created for every pooled JSON state. */ @@ -197,6 +196,9 @@ public int compare(DeclarationCandidate left, DeclarationCandidate right) { private final ConcurrentHashMap, CompletableFuture>> utf8ReaderClasses; private final ConcurrentHashMap>> utf8CollectionWriterClasses; private final ConcurrentHashMap>> utf8CollectionReaderClasses; + // Only ForyJson's fixed-pool reader-local caches publish production entries here, and each reader + // owns its configured entry limit. This reference-reuse table does not own a second capacity + // policy. private final ConcurrentHashMap cachedFieldNames; public JsonSharedRegistry(JsonConfig config) { diff --git a/java/fory-json/src/test/java/org/apache/fory/json/ForyJsonConcurrencyTest.java b/java/fory-json/src/test/java/org/apache/fory/json/ForyJsonConcurrencyTest.java new file mode 100644 index 0000000000..e1d77a8f98 --- /dev/null +++ b/java/fory-json/src/test/java/org/apache/fory/json/ForyJsonConcurrencyTest.java @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.fory.json; + +import static org.apache.fory.json.JsonTestSupport.pooledStateCount; +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertThrows; +import static org.testng.Assert.assertTrue; +import static org.testng.Assert.fail; + +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicReference; +import org.apache.fory.json.codec.AbstractJsonValueCodec; +import org.apache.fory.json.reader.JsonReader; +import org.apache.fory.json.writer.JsonWriter; +import org.testng.annotations.Test; + +public class ForyJsonConcurrencyTest { + @Test + public void configuredStateCount() { + assertEquals(pooledStateCount(ForyJson.builder().withConcurrencyLevel(3).build()), 3); + assertEquals(pooledStateCount(ForyJson.builder().withConcurrencyLevel(1).build()), 1); + assertEquals( + pooledStateCount(ForyJson.builder().build()), + Math.max(1, Runtime.getRuntime().availableProcessors() * 2)); + assertThrows(IllegalArgumentException.class, () -> ForyJson.builder().withConcurrencyLevel(0)); + } + + @Test + public void concurrencyLimitWaits() throws Exception { + CountDownLatch rootEntered = new CountDownLatch(1); + CountDownLatch releaseRoot = new CountDownLatch(1); + ForyJson json = + ForyJson.builder() + .withCodegen(false) + .withConcurrencyLevel(1) + .registerCodec(BlockingValue.class, new BlockingCodec(rootEntered, releaseRoot)) + .build(); + AtomicReference firstFailure = new AtomicReference<>(); + Thread first = + new Thread( + () -> { + try { + assertEquals(json.toJson(new BlockingValue()), "null"); + } catch (Throwable t) { + firstFailure.set(t); + } + }); + first.start(); + await(rootEntered); + + CountDownLatch secondStarted = new CountDownLatch(1); + CountDownLatch secondFinished = new CountDownLatch(1); + AtomicReference secondFailure = new AtomicReference<>(); + Thread second = + new Thread( + () -> { + secondStarted.countDown(); + try { + assertEquals(json.toJson("waiting"), "\"waiting\""); + } catch (Throwable t) { + secondFailure.set(t); + } finally { + secondFinished.countDown(); + } + }); + second.start(); + await(secondStarted); + try { + awaitAcquireContention(second); + assertEquals(secondFinished.getCount(), 1); + } finally { + releaseRoot.countDown(); + } + await(secondFinished); + first.join(); + second.join(); + assertFailure(firstFailure.get()); + assertFailure(secondFailure.get()); + } + + private static void await(CountDownLatch latch) throws InterruptedException { + assertTrue(latch.await(30, TimeUnit.SECONDS), "Timed out waiting for test coordination"); + } + + private static void awaitAcquireContention(Thread thread) { + long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(30); + while (System.nanoTime() < deadline) { + for (StackTraceElement frame : thread.getStackTrace()) { + if (frame.getClassName().equals(ForyJson.class.getName()) + && frame.getMethodName().equals("acquireContended")) { + return; + } + } + if (!thread.isAlive()) { + fail("Root operation did not wait for an execution state"); + } + Thread.yield(); + } + fail("Timed out waiting for root operation contention"); + } + + private static void assertFailure(Throwable failure) { + if (failure != null) { + fail("Unexpected worker failure", failure); + } + } + + private static final class BlockingCodec extends AbstractJsonValueCodec { + private final CountDownLatch entered; + private final CountDownLatch release; + + private BlockingCodec(CountDownLatch entered, CountDownLatch release) { + this.entered = entered; + this.release = release; + } + + @Override + public void write(JsonWriter writer, BlockingValue value) { + entered.countDown(); + try { + assertTrue(release.await(30, TimeUnit.SECONDS), "Timed out waiting to release root codec"); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new AssertionError(e); + } + writer.writeNull(); + } + + @Override + public BlockingValue read(JsonReader reader) { + reader.skipValue(); + return null; + } + } + + private static final class BlockingValue {} +} diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java index 422273a765..f4b1dc9cd7 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonAsyncCompilationTest.java @@ -527,7 +527,7 @@ public void pooledStatesRemainConcurrent() throws Exception { CountDownLatch releaseRoot = new CountDownLatch(1); CodecRegistry codecs = new CodecRegistry(); codecs.register(BlockingValue.class, new BlockingCodec(rootEntered, releaseRoot)); - ControlledJson controlled = controlledJson(codecs); + ControlledJson controlled = controlledJson(codecs, 2); AtomicReference firstFailure = new AtomicReference<>(); Thread first = new Thread( @@ -1273,6 +1273,11 @@ private static ControlledJson controlledJson() throws Exception { } private static ControlledJson controlledJson(CodecRegistry codecs) throws Exception { + return controlledJson(codecs, 1); + } + + private static ControlledJson controlledJson(CodecRegistry codecs, int concurrencyLevel) + throws Exception { JsonConfig config = new JsonConfig( false, @@ -1283,7 +1288,7 @@ private static ControlledJson controlledJson(CodecRegistry codecs) throws Except JsonAsyncCompilationTest.class.getClassLoader(), ForyJson.DEFAULT_MAX_DEPTH, ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES, - 1, + concurrencyLevel, 2 * 1024 * 1024, codecs, Collections., Class>emptyMap(), diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java index de944f4866..798c1a3b21 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonFieldNameCacheTest.java @@ -19,8 +19,9 @@ package org.apache.fory.json; +import static org.apache.fory.json.JsonTestSupport.generatedCodecId; +import static org.apache.fory.json.JsonTestSupport.generatedUtf8WriterClass; import static org.testng.Assert.assertEquals; -import static org.testng.Assert.assertNotEquals; import static org.testng.Assert.assertNotNull; import static org.testng.Assert.assertNotSame; import static org.testng.Assert.assertNull; @@ -45,27 +46,30 @@ public class JsonFieldNameCacheTest { @Test public void configuration() { - JsonConfig defaults = JsonTestSupport.config(ForyJson.builder().build()); - assertEquals(defaults.maxCachedFieldNames(), ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES); + assertEquals(ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES, 8192); assertThrows( IllegalArgumentException.class, () -> ForyJson.builder().withMaxCachedFieldNames(-1)); assertThrows( IllegalArgumentException.class, () -> ForyJson.builder().withMaxCachedFieldNames(Integer.MAX_VALUE)); - JsonConfig first = - JsonTestSupport.config( - ForyJson.builder().withConcurrencyLevel(1).withMaxCachedFieldNames(1).build()); - JsonConfig second = - JsonTestSupport.config( - ForyJson.builder().withConcurrencyLevel(1).withMaxCachedFieldNames(2).build()); - assertNotEquals(first, second); - assertNotEquals(first.hashCode(), second.hashCode()); - assertEquals(first.getCodegenHash(), second.getCodegenHash()); - assertEquals(first.maxCachedFieldNames(), 1); - assertEquals(second.maxCachedFieldNames(), 2); - assertEquals(JsonTestSupport.config(newJson(0)).maxCachedFieldNames(), 0); - assertEquals(ForyJson.DEFAULT_MAX_CACHED_FIELD_NAMES, 8192); + ForyJson oneEntry = + ForyJson.builder() + .withAsyncCompilation(false) + .withConcurrencyLevel(1) + .withMaxCachedFieldNames(1) + .build(); + ForyJson twoEntries = + ForyJson.builder() + .withAsyncCompilation(false) + .withConcurrencyLevel(1) + .withMaxCachedFieldNames(2) + .build(); + oneEntry.toJsonBytes(new TypedFields()); + twoEntries.toJsonBytes(new TypedFields()); + assertEquals( + generatedCodecId(generatedUtf8WriterClass(oneEntry, TypedFields.class)), + generatedCodecId(generatedUtf8WriterClass(twoEntries, TypedFields.class))); } @Test @@ -328,8 +332,7 @@ public void malformedNames() { @Test public void sharedHashCollision() { - JsonConfig config = JsonTestSupport.config(newJson(4)); - JsonSharedRegistry registry = new JsonSharedRegistry(config); + JsonSharedRegistry registry = JsonTestSupport.newSharedRegistry(); CachedFieldName first = registry.cacheFieldName(1L, "a", 'a', 0); CachedFieldName second = registry.cacheFieldName(1L, "b", 'b', 0); assertSame(second, first); @@ -366,8 +369,7 @@ public void readerHashCollision() { @Test public void concurrentPublication() throws Exception { - JsonConfig config = JsonTestSupport.config(newJson(1)); - JsonSharedRegistry registry = new JsonSharedRegistry(config); + JsonSharedRegistry registry = JsonTestSupport.newSharedRegistry(); int threads = 8; ExecutorService executor = Executors.newFixedThreadPool(threads); CountDownLatch start = new CountDownLatch(1); diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonGeneratedCodecTest.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonGeneratedCodecTest.java index 00c35bada6..eddd0e5a8b 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/JsonGeneratedCodecTest.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonGeneratedCodecTest.java @@ -19,7 +19,8 @@ package org.apache.fory.json; -import static org.apache.fory.json.JsonTestSupport.currentTypeResolver; +import static org.apache.fory.json.JsonTestSupport.generatedCodecId; +import static org.apache.fory.json.JsonTestSupport.generatedUtf8WriterClass; import static org.apache.fory.json.JsonTestSupport.newLatin1Reader; import static org.apache.fory.json.JsonTestSupport.newUtf8Reader; import static org.testng.Assert.assertEquals; @@ -46,7 +47,6 @@ import org.apache.fory.json.meta.JsonFieldNameHash; import org.apache.fory.json.reader.Latin1JsonReader; import org.apache.fory.json.reader.Utf8JsonReader; -import org.apache.fory.json.resolver.JsonTypeResolver; import org.testng.annotations.Test; public class JsonGeneratedCodecTest extends ForyJsonTestModels { @@ -202,10 +202,10 @@ public void sameConfigUsesSameId(boolean codegen) throws Exception { return; } - Class firstCodecClass = generatedCodecClass(first, PublicFields.class); - Class secondCodecClass = generatedCodecClass(second, PublicFields.class); - Class writeNullCodecClass = generatedCodecClass(writeNullFields, PublicFields.class); - Class snakeCaseCodecClass = generatedCodecClass(snakeCase, PublicFields.class); + Class firstCodecClass = generatedUtf8WriterClass(first, PublicFields.class); + Class secondCodecClass = generatedUtf8WriterClass(second, PublicFields.class); + Class writeNullCodecClass = generatedUtf8WriterClass(writeNullFields, PublicFields.class); + Class snakeCaseCodecClass = generatedUtf8WriterClass(snakeCase, PublicFields.class); assertEquals(firstCodecClass.getPackage().getName(), PublicFields.class.getPackage().getName()); assertEquals( secondCodecClass.getPackage().getName(), PublicFields.class.getPackage().getName()); @@ -213,9 +213,9 @@ public void sameConfigUsesSameId(boolean codegen) throws Exception { assertGeneratedName(secondCodecClass, PublicFields.class, "Utf8Writer"); assertGeneratedName(writeNullCodecClass, PublicFields.class, "Utf8Writer"); assertGeneratedName(snakeCaseCodecClass, PublicFields.class, "Utf8Writer"); - assertEquals(generatedId(secondCodecClass), generatedId(firstCodecClass)); - assertNotEquals(generatedId(writeNullCodecClass), generatedId(firstCodecClass)); - assertNotEquals(generatedId(snakeCaseCodecClass), generatedId(firstCodecClass)); + assertEquals(generatedCodecId(secondCodecClass), generatedCodecId(firstCodecClass)); + assertNotEquals(generatedCodecId(writeNullCodecClass), generatedCodecId(firstCodecClass)); + assertNotEquals(generatedCodecId(snakeCaseCodecClass), generatedCodecId(firstCodecClass)); } @Test @@ -306,7 +306,7 @@ public void writeSplitGeneratedFields() throws Exception { expected.append('}'); assertEquals(new String(json.toJsonBytes(value), StandardCharsets.UTF_8), expected.toString()); - Class generated = generatedCodecClass(json, WideWriterFields.class); + Class generated = generatedUtf8WriterClass(json, WideWriterFields.class); int groups = 0; for (Method method : generated.getDeclaredMethods()) { if (method.getName().startsWith("writeUtf8Group")) { @@ -446,28 +446,12 @@ private static void assertObjectCollections(ObjectCollections value, String name assertEquals(value.set.iterator().next().name, name + 10); } - private static Class generatedCodecClass(ForyJson json, Class type) throws Exception { - JsonTypeResolver typeResolver = currentTypeResolver(json); - Object owner = typeResolver.getObjectCodec(type); - Object codec = typeResolver.getTypeInfo(type, type).utf8Writer(); - assertTrue(codec != owner, codec.getClass().getName()); - return codec.getClass(); - } - private static void assertGeneratedName( Class generatedClass, Class valueType, String role) { String simpleName = generatedClass.getSimpleName(); assertTrue(simpleName.startsWith(valueType.getSimpleName()), generatedClass.getName()); assertTrue(simpleName.contains(role + GENERATED_SUFFIX), generatedClass.getName()); assertFalse(simpleName.contains(GENERATED_SUFFIX + "_"), generatedClass.getName()); - assertTrue(generatedId(generatedClass) >= 0, generatedClass.getName()); - } - - private static int generatedId(Class generatedClass) { - String simpleName = generatedClass.getSimpleName(); - int suffixStart = simpleName.lastIndexOf(GENERATED_SUFFIX); - assertTrue(suffixStart >= 0, generatedClass.getName()); - String id = simpleName.substring(suffixStart + GENERATED_SUFFIX.length()); - return id.isEmpty() ? 0 : Integer.parseInt(id); + assertTrue(generatedCodecId(generatedClass) >= 0, generatedClass.getName()); } } diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonMixinTest.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonMixinTest.java index b84d7605fe..a3c17dcbfd 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/JsonMixinTest.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonMixinTest.java @@ -19,6 +19,8 @@ package org.apache.fory.json; +import static org.apache.fory.json.JsonTestSupport.generatedCodecId; +import static org.apache.fory.json.JsonTestSupport.generatedUtf8WriterClass; import static org.testng.Assert.assertEquals; import static org.testng.Assert.assertNotEquals; import static org.testng.Assert.assertThrows; @@ -189,14 +191,20 @@ public void registrationLifecycle() { .registerMixin(FirstNameMixin.class) .build(); ForyJson equivalent = newJsonBuilder().registerMixin(FirstNameMixin.class).build(); - assertEquals(JsonTestSupport.config(repeated), JsonTestSupport.config(equivalent)); - assertEquals( - JsonTestSupport.config(repeated).getCodegenHash(), - JsonTestSupport.config(equivalent).getCodegenHash()); - assertNotEquals(JsonTestSupport.config(first), JsonTestSupport.config(second)); - assertNotEquals( - JsonTestSupport.config(first).getCodegenHash(), - JsonTestSupport.config(second).getCodegenHash()); + assertEquals(repeated.toJson(new NameTarget("repeat")), "{\"first\":\"repeat\"}"); + assertEquals(equivalent.toJson(new NameTarget("equal")), "{\"first\":\"equal\"}"); + if (codegenEnabled()) { + first.toJsonBytes(new NameTarget("first")); + second.toJsonBytes(new NameTarget("second")); + repeated.toJsonBytes(new NameTarget("repeat")); + equivalent.toJsonBytes(new NameTarget("equal")); + assertEquals( + generatedCodecId(generatedUtf8WriterClass(repeated, NameTarget.class)), + generatedCodecId(generatedUtf8WriterClass(equivalent, NameTarget.class))); + assertNotEquals( + generatedCodecId(generatedUtf8WriterClass(first, NameTarget.class)), + generatedCodecId(generatedUtf8WriterClass(second, NameTarget.class))); + } assertGeneratedWhenSupported(first, NameTarget.class); assertGeneratedWhenSupported(second, NameTarget.class); } diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonStringTest.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonStringTest.java index cd735f4fb8..e898c9395b 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/JsonStringTest.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonStringTest.java @@ -23,7 +23,6 @@ import static org.apache.fory.json.JsonTestSupport.newLatin1Reader; import static org.apache.fory.json.JsonTestSupport.newStringWriter; import static org.apache.fory.json.JsonTestSupport.newUtf16Reader; -import static org.apache.fory.json.JsonTestSupport.pooledStateCount; import static org.testng.Assert.assertEquals; import static org.testng.Assert.assertFalse; import static org.testng.Assert.assertThrows; @@ -234,16 +233,14 @@ public void stringWriterUtf16Escapes() { } @Test - public void builderRuntimeLimits() throws Exception { + public void writerBufferLimit() throws Exception { int bufferLimit = 64 * 1024; ForyJson json = ForyJson.builder() .withAsyncCompilation(false) - .withConcurrencyLevel(3) + .withConcurrencyLevel(1) .withBufferSizeLimitBytes(bufferLimit) .build(); - assertEquals(pooledStateCount(json), 3); - assertEquals(pooledStateCount(ForyJson.builder().withConcurrencyLevel(1).build()), 1); String value = repeat('a', bufferLimit + 1); StringJsonWriter stringWriter = (StringJsonWriter) currentStateField(json, "stringWriter"); @@ -258,12 +255,9 @@ public void builderRuntimeLimits() throws Exception { utf8Writer.reset(); assertEquals(writerBufferLength(utf8Writer), bufferLimit); - JsonConfig defaultConfig = JsonTestSupport.config(ForyJson.builder().build()); - assertEquals( - defaultConfig.concurrencyLevel(), - Math.max(1, Runtime.getRuntime().availableProcessors() * 2)); - assertEquals(defaultConfig.bufferSizeLimitBytes(), 2 * 1024 * 1024); - assertThrows(IllegalArgumentException.class, () -> ForyJson.builder().withConcurrencyLevel(0)); + ForyJson defaults = ForyJson.builder().build(); + assertEquals(writerBufferLimit(currentStateField(defaults, "stringWriter")), 2 * 1024 * 1024); + assertEquals(writerBufferLimit(currentStateField(defaults, "utf8Writer")), 2 * 1024 * 1024); assertThrows( IllegalArgumentException.class, () -> ForyJson.builder().withBufferSizeLimitBytes(0)); } @@ -555,6 +549,12 @@ private static int writerBufferLength(Object writer) throws Exception { return ((byte[]) field.get(writer)).length; } + private static int writerBufferLimit(Object writer) throws Exception { + Field field = writer.getClass().getDeclaredField("bufferSizeLimitBytes"); + field.setAccessible(true); + return field.getInt(writer); + } + private static int readerBufferLength(Object reader) throws Exception { Field field = reader.getClass().getDeclaredField("stringDecodeBuffer"); field.setAccessible(true); diff --git a/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java b/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java index eed12803a3..c01f34bcf2 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/JsonTestSupport.java @@ -34,6 +34,7 @@ import org.apache.fory.serializer.StringSerializer; final class JsonTestSupport { + private static final String GENERATED_CODEC_SUFFIX = "ForyJsonCodec"; private static final JsonConfig CONFIG = new JsonConfig( false, @@ -166,12 +167,28 @@ static Object pooledStateField(ForyJson json, int index, String name) { } } - static JsonConfig config(ForyJson json) { - try { - return (JsonConfig) field(json, "config"); - } catch (ReflectiveOperationException e) { - throw new AssertionError(e); + static JsonSharedRegistry newSharedRegistry() { + return new JsonSharedRegistry(CONFIG); + } + + static Class generatedUtf8WriterClass(ForyJson json, Class type) { + JsonTypeResolver resolver = currentTypeResolver(json); + Object owner = resolver.getObjectCodec(type); + Object codec = resolver.getTypeInfo(type, type).utf8Writer(); + if (codec == owner) { + throw new AssertionError("No generated UTF-8 writer for " + type.getName()); + } + return codec.getClass(); + } + + static int generatedCodecId(Class generatedClass) { + String simpleName = generatedClass.getSimpleName(); + int suffixStart = simpleName.lastIndexOf(GENERATED_CODEC_SUFFIX); + if (suffixStart < 0) { + throw new AssertionError("Unexpected generated class " + generatedClass.getName()); } + String id = simpleName.substring(suffixStart + GENERATED_CODEC_SUFFIX.length()); + return id.isEmpty() ? 0 : Integer.parseInt(id); } static String stringReaderPath(String input) { diff --git a/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java b/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java index 5b164b1b51..826f031126 100644 --- a/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java +++ b/java/fory-json/src/test/java/org/apache/fory/json/reader/FieldNameCacheTest.java @@ -29,6 +29,7 @@ import org.apache.fory.json.meta.JsonFieldNameHash; import org.apache.fory.json.resolver.JsonSharedRegistry; import org.apache.fory.json.resolver.JsonSharedRegistry.CachedFieldName; +import org.apache.fory.json.resolver.JsonTypeResolver; import org.testng.annotations.Test; public class FieldNameCacheTest { @@ -81,11 +82,17 @@ public void idempotentPut() { private static JsonSharedRegistry registry() { try { - ForyJson json = ForyJson.builder().withCodegen(false).build(); - Field field = ForyJson.class.getDeclaredField("sharedRegistry"); - field.setAccessible(true); - return (JsonSharedRegistry) field.get(json); - } catch (NoSuchFieldException | IllegalAccessException e) { + ForyJson json = ForyJson.builder().withCodegen(false).withConcurrencyLevel(1).build(); + Field slotsField = ForyJson.class.getDeclaredField("slots"); + slotsField.setAccessible(true); + Object stateSlot = ((Object[]) slotsField.get(json))[0]; + Field stateField = stateSlot.getClass().getDeclaredField("state"); + stateField.setAccessible(true); + Object state = stateField.get(stateSlot); + Field resolverField = state.getClass().getDeclaredField("typeResolver"); + resolverField.setAccessible(true); + return ((JsonTypeResolver) resolverField.get(state)).sharedRegistry(); + } catch (ReflectiveOperationException e) { throw new AssertionError(e); } }