Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix immutable collection or map fields deserialization issue. #260

Merged
merged 6 commits into from Jan 29, 2019

Conversation

neoremind
Copy link

What changes were proposed in this pull request?

Immutable collection or map fields usually does not support operations like add/set/remove, for example java.util.HashMap$KeySet, com.google.common.collect.ImmutableCollection, one solution is to create Delegate for the specific type to do custom mergeFrom and writeTo action.

In DefaultIdStrategy class, the registerDelegate method relies on typeClass() provided by Delegate to get the class name.

public <T> boolean registerDelegate(Delegate<T> delegate)
{
    return null == delegateMapping.putIfAbsent(delegate.typeClass()
            .getName(), new HasDelegate<T>(delegate, this));
}

But in some cases where class name can not be initiated like java.util.HashMap$KeySet, because the inner class is not visible since the class modifier is default.

So there should be a way to set specific class name manually. This PR add another registerDelegate method for people to specify class name.

Moreover, this PR will solve the issue #227. If one collection is not inlined and cannot be found by CollectionSchema$MessageFactories, the following exception will be thrown.

java.lang.RuntimeException: java.lang.IllegalArgumentException: No enum constant io.protostuff.CollectionSchema.MessageFactories.KeySet
	at io.protostuff.runtime.RuntimeMapFieldFactory$5.mergeFrom(RuntimeMapFieldFactory.java:508)
	at io.protostuff.runtime.RuntimeSchema.mergeFrom(RuntimeSchema.java:466)
	at io.protostuff.IOUtil.mergeFrom(IOUtil.java:46)
	at io.protostuff.ProtobufIOUtil.mergeFrom(ProtobufIOUtil.java:103)
	at io.protostuff.ProtoCodecUtil.decode(ProtoCodecUtil.java:33)
	at io.protostuff.ut.PersonTest.test_2(PersonTest.java:91)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.IllegalArgumentException: No enum constant io.protostuff.CollectionSchema.MessageFactories.KeySet
	at java.lang.Enum.valueOf(Enum.java:238)
	at io.protostuff.CollectionSchema$MessageFactories.valueOf(CollectionSchema.java:53)
	at io.protostuff.runtime.DefaultIdStrategy.resolveCollectionFrom(DefaultIdStrategy.java:349)
	at io.protostuff.runtime.ObjectSchema.readObjectFrom(ObjectSchema.java:581)
	at io.protostuff.runtime.ObjectSchema.mergeFrom(ObjectSchema.java:350)
	at io.protostuff.ByteArrayInput.mergeObject(ByteArrayInput.java:503)
	at io.protostuff.runtime.RuntimeMapFieldFactory$5.vPutFrom(RuntimeMapFieldFactory.java:563)
	at io.protostuff.runtime.RuntimeMapField$1.putValueFrom(RuntimeMapField.java:61)
	at io.protostuff.MapSchema$2.mergeFrom(MapSchema.java:503)
	at io.protostuff.MapSchema$2.mergeFrom(MapSchema.java:398)
	at io.protostuff.ByteArrayInput.mergeObject(ByteArrayInput.java:503)
	at io.protostuff.MapSchema.mergeFrom(MapSchema.java:344)
	at io.protostuff.MapSchema.mergeFrom(MapSchema.java:31)
	at io.protostuff.ByteArrayInput.mergeObject(ByteArrayInput.java:503)
	at io.protostuff.runtime.RuntimeMapFieldFactory$5.mergeFrom(RuntimeMapFieldFactory.java:503)
	... 27 more

This PR also solves this issue by checking whether the collection or map class can be found in CollectionSchema$MessageFactories or MapSchema$MessageFactories, if not the collection id or map id will use the full class name instead of simple class name.

How was this patch tested?

Add two test cases in AbstractRuntimeObjectSchemaTest in protosutff-runtime module.

One to check java.util.HashMap$KeySet can be ser and deser successfully.

One to check customized immutable list can be ser and deser successfully.

@neoremind
Copy link
Author

neoremind commented Jan 29, 2019

@dyu Would you help to review this PR? The change is small and focused, and I have added test cases to verify the specific issue. Many thanks.

}
catch (IllegalArgumentException e)
{
return false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an expensive way to check if it exists ... if there's no other way, then this will probably do

Copy link
Author

@neoremind neoremind Jan 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Iterating the enums is probably heavy. How about caching it as static?

    public static boolean accept(String name)
    {
        return MESSAGE_FACTORIES_NAMES.contains(name);
    }
    
    static Set<String> MESSAGE_FACTORIES_NAMES;
    
    static 
    {
        MessageFactories[] messageFactories = MessageFactories.values();
        MESSAGE_FACTORIES_NAMES = new HashSet<String>(messageFactories.length);
        for (MessageFactories messageFactory : messageFactories) {
            MESSAGE_FACTORIES_NAMES.add(messageFactory.name());
        }
    }

Copy link
Author

@neoremind neoremind Jan 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the PR by using a less expensive way. Please check.

* This is used by {@link MessageFactories#accept(String)} method. Rather than iterating enums in runtime
* which can be an expensive way to do, caching all the enums as static property will be a good way.
*/
static Set<String> MESSAGE_FACTORIES_NAMES;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mark as final

* This is used by {@link MapSchema.MessageFactories#accept(String)} method. Rather than iterating enums in runtime
* which can be an expensive way to do, caching all the enums as static property will be a good way.
*/
static Set<String> MESSAGE_FACTORIES_NAMES;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mark as final

Copy link
Author

@neoremind neoremind Jan 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants