[Java] Auto serializer selection falls back to JDK-compatible path for sorted container subclasses

[mandrean]

Search before asking

• I had searched in the issues and found no similar issues.

Version

v0.16.0

Component(s)

Java

Minimal reproduce step

This is similar to #3337 / #3342 and #3343 / #3344, but the bug here is in the auto-selection path.

A minimal reproducer is:

package org.apache.fory.serializer.collection;

import static org.testng.Assert.assertEquals;

import java.util.Arrays;
import java.util.TreeSet;
import org.apache.fory.Fory;
import org.apache.fory.config.CompatibleMode;
import org.apache.fory.config.Language;
import org.apache.fory.serializer.Serializer;
import org.testng.annotations.Test;

public class TreeSetAutoSelectionReproTest {
  public static class ChildTreeSet extends TreeSet<String> {
    public String state;

    public ChildTreeSet() {}
  }

  @Test
  public void testAutoSelectionFallsBackToJdkCompatibleSerializer() {
    Fory fory =
        Fory.builder()
            .withLanguage(Language.JAVA)
            .requireClassRegistration(false)
            .withRefTracking(true)
            .withCompatibleMode(CompatibleMode.COMPATIBLE)
            .build();

    Serializer<?> serializer = fory.getTypeResolver().getSerializer(ChildTreeSet.class);
    assertEquals(
        serializer.getClass(), CollectionSerializers.JDKCompatibleCollectionSerializer.class);

    ChildTreeSet values = new ChildTreeSet();
    values.state = "state";
    values.addAll(Arrays.asList("b", "a"));

    ChildTreeSet roundTripped = (ChildTreeSet) fory.deserialize(fory.serialize(values));
    assertEquals(roundTripped, values);
    assertEquals(roundTripped.state, values.state);
  }
}

What did you expect to see?

For an eligible TreeSet subclass like the one above, Fory should auto-select the optimized child-container serializer path rather than falling back to JDK-compatible serialization.

In other words, it should behave like the explicit/manual serializer path and avoid ObjectStreamSerializer entirely.

What did you see instead?

Auto selection picks:

CollectionSerializers.JDKCompatibleCollectionSerializer

and logs:

... customized jdk serialization, which is inefficient ...

This means the class is routed through ObjectStreamSerializer / meta-shared compatible serialization instead of the optimized child-container path.

Anything Else?

• I verified the minimal reproducer above on clean main and it round-trips successfully, but it still selects the wrong serializer and emits the warning.
• This is not limited to TreeSet subclasses. The same auto-selection gap also affects eligible subclasses of:
  • TreeMap
  • ConcurrentSkipListSet
  • ConcurrentSkipListMap
  • PriorityQueue
• Constructor-only wrappers such as (Map), (SortedMap), (Collection), (SortedSet), and comparator-based constructors are particularly relevant here.
• This is separate from the async layer-JIT bug fixed in #3514: async compilation made the fallback path more painful, but the wrong auto-selection happens even without async compilation.
• Manual registration is already better here after #3344; the missing piece is making the auto path use the same constructor-aware optimized serializers.

Are you willing to submit a PR?

• I'm willing to submit a PR!