Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kryo 序列化 null 的问题~ #356

Open
Gaion opened this issue Aug 30, 2017 · 1 comment
Open

kryo 序列化 null 的问题~ #356

Gaion opened this issue Aug 30, 2017 · 1 comment

Comments

@Gaion
Copy link

Gaion commented Aug 30, 2017 •
edited

使用过程中，发现一个问题，直连情况下参数传 null 正常，但是从注册中心走一遍，报错~

api

public interface DemoService {
    void show(Long a, Long b, String c);
}

提供方

public class DemoServiceImpl implements DemoService {
    public void show(Long a, Long b, String c) {
        System.out.println(StringUtils.format("a:{};b:{},c:{}", a, b, c));
    }
}

    <dubbo:application name="bsm-support"/>
    <dubbo:protocol name="dubbo" port="20880" serialization="kryo"/>
    <dubbo:registry address="zookeeper://127.0.0.1:2181" />
    <dubbo:annotation package="com.demo.bsm.support" />
    <dubbo:provider timeout="10000" retries="0"/>

消费者

public void show() {
    demoService.show(1L, 2L, "c");
    demoService.show(1L, 2L, null);
    demoService.show(1L, null, "c");
}

消费者直连XML

    <dubbo:application name="supervise-web"/>
    <dubbo:consumer registry="N/A" timeout="10000"/>
    <dubbo:annotation package="com.demo.supervise.web" />

调用后提供方输出

a:1;b:2,c:c
a:1;b:2,c:
a:1;b:,c:c

消费者通过注册中心的XML

    <dubbo:application name="supervise-web"/>
    <dubbo:registry address="zookeeper://127.0.0.1:2181"/>
    <dubbo:annotation package="com.demo.supervise.web" />

调用后提供方输出

a:1;b:2,c:c

提供方报错

15:44:13.340 [New I/O worker #1] WARN com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation - [DUBBO] Decode rpc invocation failed: java.lang.String cannot be cast to java.util.Map, dubbo version: 2.8.4, current host: 10.37.129.2
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode(DecodeableRpcInvocation.java:150) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation.decode(DecodeableRpcInvocation.java:74) [dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCodec.decodeBody(DubboCodec.java:138) [dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:134) [dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:95) [dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCountCodec.decode(DubboCountCodec.java:46) [dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.remoting.transport.netty.NettyCodecAdapter$InternalDecoder.messageReceived(NettyCodecAdapter.java:134) [dubbo-2.8.4.jar:2.8.4]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.7.0.Final.jar:?]
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.7.0.Final.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_75]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_75]

消费者报错

15:16:29.392 [http-bio-8081-exec-1] ERROR com.demo.common.bean.web.ExceptionHandler - Failed to invoke the method show in the service com.demo.bsm.support.api.service.DemoService. Tried 1 times of the providers [10.37.129.2:20880] (1/1) from the registry 127.0.0.1:2181 on the consumer 10.37.129.2 using the dubbo version 2.8.4. Last error is: Failed to invoke remote method: show, provider: dubbo://10.37.129.2:20880/com.demo.bsm.support.api.service.DemoService?anyhost=true&application=supervise-web&check=false&default.retries=0&default.timeout=10000&dubbo=2.8.4&generic=false&interface=com.demo.bsm.support.api.service.DemoService&methods=show&pid=3686&serialization=kryo&side=consumer&timestamp=1504077312779, cause: Fail to decode request due to: RpcInvocation [methodName=show, parameterTypes=[], arguments=null, attachments={dubbo=2.8.4, input=242, path=com.demo.bsm.support.api.service.DemoService, version=0.0.0}]
com.alibaba.dubbo.rpc.RpcException: Failed to invoke the method show in the service com.demo.bsm.support.api.service.DemoService. Tried 1 times of the providers [10.37.129.2:20880] (1/1) from the registry 127.0.0.1:2181 on the consumer 10.37.129.2 using the dubbo version 2.8.4. Last error is: Failed to invoke remote method: show, provider: dubbo://10.37.129.2:20880/com.demo.bsm.support.api.service.DemoService?anyhost=true&application=supervise-web&check=false&default.retries=0&default.timeout=10000&dubbo=2.8.4&generic=false&interface=com.demo.bsm.support.api.service.DemoService&methods=show&pid=3686&serialization=kryo&side=consumer&timestamp=1504077312779, cause: Fail to decode request due to: RpcInvocation [methodName=show, parameterTypes=[], arguments=null, attachments={dubbo=2.8.4, input=242, path=com.demo.bsm.support.api.service.DemoService, version=0.0.0}]
at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:101) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:227) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:72) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:52) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.common.bytecode.proxy2.show(proxy2.java) ~[dubbo-2.8.4.jar:2.8.4]
at com.demo.supervise.web.service.ShopService.show(ShopService.java:36) ~[ShopService.class:?]
at com.demo.supervise.web.controller.ShopController.regionDialog(ShopController.java:151) ~[ShopController.class:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.7.0_75]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[?:1.7.0_75]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.7.0_75]
at java.lang.reflect.Method.invoke(Method.java:606) ~[?:1.7.0_75]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620) [servlet-api.jar:?]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [catalina.jar:7.0.57]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat7-websocket.jar:7.0.57]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]
at org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter.doFilterInternal(OpenEntityManagerInViewFilter.java:178) [spring-orm-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90) [shiro-core-1.2.4.jar:1.2.4]
at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83) [shiro-core-1.2.4.jar:1.2.4]
at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383) [shiro-core-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362) [shiro-web-1.2.4.jar:1.2.4]
at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125) [shiro-web-1.2.4.jar:1.2.4]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262) [spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [catalina.jar:7.0.57]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [catalina.jar:7.0.57]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:503) [catalina.jar:7.0.57]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) [catalina.jar:7.0.57]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) [catalina.jar:7.0.57]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) [catalina.jar:7.0.57]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [catalina.jar:7.0.57]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) [catalina.jar:7.0.57]
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) [tomcat-coyote.jar:7.0.57]
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) [tomcat-coyote.jar:7.0.57]
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316) [tomcat-coyote.jar:7.0.57]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_75]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_75]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.57]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_75]
Caused by: com.alibaba.dubbo.remoting.RemotingException: Fail to decode request due to: RpcInvocation [methodName=show, parameterTypes=[], arguments=null, attachments={dubbo=2.8.4, input=242, path=com.demo.bsm.support.api.service.DemoService, version=0.0.0}]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.returnFromResponse(DefaultFuture.java:190) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:110) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:84) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.dubbo.DubboInvoker.doInvoke(DubboInvoker.java:96) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:144) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:74) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:53) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:75) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:48) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:53) ~[dubbo-2.8.4.jar:2.8.4]
at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:77) ~[dubbo-2.8.4.jar:2.8.4]
... 63 more
@Gaion
Copy link
Author

Gaion commented Aug 30, 2017

翻了半天源码，发现，使用 kyro、fst 进行序列化时，参数为 null 的情况下，就不写入参数的数量，导致在反序列化的时候由于获取不到参数数量而报错，默认的序列化访问正常。问一下，原来为啥设计成这样？是 BUG，还是有特殊考虑？

序列化时的代码，com.alibaba.dubbo.rpc.protocol.dubbo.DubboCodec的encodeRequestData方法

protected void encodeRequestData(Channel channel, ObjectOutput out, Object data) throws IOException {
    .....
    if (getSerialization(channel) instanceof OptimizedSerialization) {  //kyro、fst序列化
        if (!containComplexArguments(inv)) { //写入参数数量
            out.writeInt(inv.getParameterTypes().length);
        } else { //无参数
            out.writeInt(-1);
        }
    } else { //其他序列化方式
        out.writeUTF(ReflectUtils.getDesc(inv.getParameterTypes()));
    }
    .....
}
private boolean containComplexArguments(RpcInvocation invocation) {
    for (int i = 0; i < invocation.getParameterTypes().length; i++) {
        if (invocation.getArguments()[i] == null || invocation.getParameterTypes()[i] != invocation.getArguments()[i].getClass()) { //参数为 null，或者实际参数类型与定义不一致？
            return true;
        }
    }
    return false;
}

反序列化时的代码，com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcInvocation 的 decode 方法

public Object decode(Channel channel, InputStream input) throws IOException {
    .....
    int argNum = -1;
    if (CodecSupport.getSerialization(channel.getUrl(), serializationType) instanceof OptimizedSerialization) { //kyro、fst 序列化时，读取参数数量
        argNum = in.readInt();
    }
    if (argNum >= 0) { // 包含参数的处理
        ...
    } else { //没有读取到参数，则认为是其他序列化
        ...
    }
    ...
}

devenfan pushed a commit to devenfan/dubbox that referenced this issue Sep 10, 2017
@qinliujie
Merge pull request dangdangdotcom#356 from zssazrael/patch-1
5e5b9e6 
fix:修复 com.alibaba.dubbo.rpc.protocol.memcached.MemcachedProtocol 配置文件类名错误
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Gaion