Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nacos 2.0.4 Cluster mode When Skywalking Agent is used, services cannot be registered #8275

Closed
ZhengBing520 opened this issue Apr 28, 2022 · 11 comments
Labels
area/Client Related to Nacos Client SDK priority/high Very important, need to be worked with soon but not very urgent
Milestone

Comments

@ZhengBing520
Copy link
Contributor

Describe the bug
Nacos 2.0.4 Cluster mode When Skywalking Agent is used, services cannot be registered
nacos.logoutput error log: java.lang.ClassCastException: io.grpc.ClientInterceptors$InterceptorChannel cannot be cast to io.grpc.ManagedChannel
cause:

  1. skywalking agent:enhance io.grpc.stub.AbstractStub#getChannel,convert ClientInterceptors.InterceptorChannel
  2. nacos server:grpcConn.setChannel((ManagedChannel) newChannelStubTemp.getChannel()); (GrpcClient.java:281)

Expected behavior
Normal Running .

Acutally behavior
Demo registration and communication are affected. Services can be found from the service list in the Nacos console, but an error is reported when editing the service
(影响demo注册和通信,可以从nacos控制台的服务列表中发现服务,但是编辑服务时报错,报错位置如下)
ServiceManager#updateInstance -> throw new NacosException(NacosException.INVALID_PARAM, "instance not exist: " + instance);
This indicates that the instance was not registered successfully

How to Reproduce
Steps to reproduce the behavior:

  1. Running cluster Mode(3 nodes): -Dserver.port -Dnacos.home -Dnacos.logs.path -Dnacos.standalone=false
  2. Agent is configured for both the NACOS service and Demo: -javaagent:/skywalking/skywalking-agent/skywalking-agent.jar
  3. View the log from nacos.log
  4. See error:
    java.lang.ClassCastException: io.grpc.ClientInterceptors$InterceptorChannel cannot be cast to io.grpc.ManagedChannel
    at com.alibaba.nacos.common.remote.client.grpc.GrpcClient.connectToServer(GrpcClient.java:281)
    at com.alibaba.nacos.common.remote.client.RpcClient.reconnect(RpcClient.java:522)
    at com.alibaba.nacos.common.remote.client.RpcClient.lambda$start$2(RpcClient.java:370)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    2022-04-28 12:58:42,189 INFO [Cluster-192.168.90.75:8868] Fail to connect server, after trying 315 times, last try server is {serverIp = '192.168.90.75', server main port = 8868}, error = unknown

Desktop (please complete the following information):

  • OS: Centos、Windows
  • Version nacos-server 2.0.4, nacos-client 2.0.4
  • Module naming (console)
  • SDK spring-cloud-alibaba-nacos + nacos-client 2.0.4 + skywalking agent 8.7

Additional context

@ZhengBing520
Copy link
Contributor Author

The first time I submitted it, my PR was stuck. Could you give me some help

@bangbang2333
Copy link
Contributor

帮顶

ZhengBing520 added a commit to ZhengBing520/nacos that referenced this issue Apr 29, 2022
…rs$InterceptorChannel cannot be cast to io.grpc.ManagedChannel
@realJackSun realJackSun added area/Client Related to Nacos Client SDK priority/high Very important, need to be worked with soon but not very urgent labels May 6, 2022
@Oliverwqcwrw
Copy link
Contributor

Oliverwqcwrw commented May 11, 2022

Hello, @ZhengBing520
Is the agent also specified when the cluster is started?
Whether the cluster uses embedded datasource or external datasource?

@ZhengBing520
Copy link
Contributor Author

Hello, @ZhengBing520 Is the agent also specified when the cluster is started?
Yes, the cluster is connected to the Agent to get the complete tracking data

Whether the cluster uses embedded datasource or external datasource?
external datasource,using MySQL

@ZhengBing520
Copy link
Contributor Author

hello, @Oliverwqcwrw
sound code :grpcConn.setChannel((ManagedChannel) newChannelStubTemp.getChannel()); (GrpcClient.java:281)
We can tweak the code so that instead of going directly to ManagedChannel
Please take a look at this pr: #8276
Thank you

@Oliverwqcwrw
Copy link
Contributor

I am trying to reproduce this problem. If it does, I will see if your PR solves the problem

@ZhengBing520
Copy link
Contributor Author

I am trying to reproduce this problem. If it does, I will see if your PR solves the problem

Thanks for you

@Oliverwqcwrw
Copy link
Contributor

hello, @Oliverwqcwrw sound code :grpcConn.setChannel((ManagedChannel) newChannelStubTemp.getChannel()); (GrpcClient.java:281) We can tweak the code so that instead of going directly to ManagedChannel Please take a look at this pr: #8276 Thank you

I have reproduced the problem and verified your PR, it is a good discovery. Thank you for your contribution to the community. It will be merged after @shiyiyue1102 review passed

@ZhengBing520
Copy link
Contributor Author

Sorry, My misoperation

@ZhengBing520 ZhengBing520 reopened this May 11, 2022
@ZhengBing520
Copy link
Contributor Author

hello, @Oliverwqcwrw sound code :grpcConn.setChannel((ManagedChannel) newChannelStubTemp.getChannel()); (GrpcClient.java:281) We can tweak the code so that instead of going directly to ManagedChannel Please take a look at this pr: #8276 Thank you

I have reproduced the problem and verified your PR, it is a good discovery. Thank you for your contribution to the community. It will be merged after @shiyiyue1102 review passed

Thanks for you. This is the first issue I opened, I am not familiar with the process

@Oliverwqcwrw
Copy link
Contributor

It doesn't matter. You can talk about any problems. The community is very friendly

ZhengBing520 added a commit to ZhengBing520/nacos that referenced this issue May 11, 2022
…rs$InterceptorChannel cannot be cast to io.grpc.ManagedChannel
ZhengBing520 added a commit to ZhengBing520/nacos that referenced this issue May 23, 2022
…rs$InterceptorChannel cannot be cast to io.grpc.ManagedChannel
KomachiSion pushed a commit that referenced this issue May 23, 2022
…rceptorChannel cannot be cast to io.grpc.ManagedChannel (#8374)
@KomachiSion KomachiSion added this to the 2.1.1 milestone May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/Client Related to Nacos Client SDK priority/high Very important, need to be worked with soon but not very urgent
Projects
None yet
Development

No branches or pull requests

5 participants