dubbo-xds not support WeightedCluster #271

AYue-94 · 2023-05-31T12:56:36Z

Environment

Dubbo version: 3.2.0
Operating System version: xxx
Java version: 1.8

Steps to reproduce this issue

1）use dubbo-samples/3-extensions/registry/dubbo-samples-xds
2）kubectl label pods -n dubbo-demo {podName1} version=v1 && kubectl label pods -n dubbo-demo {podName2} version=v2
3）kubectl apply -f VirtualService.yaml

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: dubbo-samples-xds-provider
  namespace: dubbo-demo
spec:
  hosts:
    - dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
  http:
    - match:
        - port: 50051
      route:
        - destination:
            host: dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
            subset: v1
            port:
              number: 50051
          weight: 50
        - destination:
            host: dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
            subset: v2
            port:
              number: 50051
          weight: 50

---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: dubbo-samples-xds-provider
  namespace: dubbo-demo
spec:
  host: dubbo-samples-xds-provider.dubbo-demo.svc.cluster.local
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

Expected Behavior

consumer can find 2 invokers.

Actual Behavior

no invokers found.

May 31, 2023 5:47:26 AM org.apache.dubbo.rpc.cluster.directory.AbstractDirectory warn
WARNING:  [DUBBO] No provider available after connectivity filter for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 All validInvokers' size: 0 All routed invokers' size: 0 All invokers' size: 0 from registry istiod.istio-system.svc:15012 on the consumer 10.1.3.76 using the dubbo version 3.2.0., dubbo version: 3.2.0, current host: 10.1.3.76, error code: 2-2. This may be caused by provider server or registry center crashed, go to https://dubbo.apache.org/faq/2/2 to find instructions.
org.apache.dubbo.rpc.RpcException: Failed to invoke the method sayHello in the service org.apache.dubbo.samples.api.GreetingService. No provider available for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 from registry istiod.istio-system.svc:15012 on the consumer 10.1.3.76 using the dubbo version 3.2.0. Please check if the providers have been started and registered.
	at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.checkInvokers(AbstractClusterInvoker.java:367)
	at org.apache.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:60)
	at org.apache.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:341)
	at org.apache.dubbo.rpc.cluster.router.RouterSnapshotFilter.invoke(RouterSnapshotFilter.java:46)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:101)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.rpc.cluster.filter.support.MetricsClusterFilter.invoke(MetricsClusterFilter.java:51)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:52)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.rpc.cluster.filter.support.ObservationSenderFilter.invoke(ObservationSenderFilter.java:61)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.spring.security.filter.ContextHolderParametersSelectedTransferFilter.invoke(ContextHolderParametersSelectedTransferFilter.java:41)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.rpc.cluster.filter.support.ConsumerClassLoaderFilter.invoke(ConsumerClassLoaderFilter.java:40)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.rpc.cluster.filter.support.ConsumerContextFilter.invoke(ConsumerContextFilter.java:118)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:331)
	at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:194)
	at org.apache.dubbo.rpc.cluster.support.wrapper.AbstractCluster$ClusterFilterInvoker.invoke(AbstractCluster.java:91)
	at org.apache.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:103)
	at org.apache.dubbo.rpc.cluster.support.wrapper.ScopeClusterInvoker.invoke(ScopeClusterInvoker.java:131)
	at org.apache.dubbo.registry.client.migration.MigrationInvoker.invoke(MigrationInvoker.java:284)
	at org.apache.dubbo.rpc.proxy.InvocationUtil.invoke(InvocationUtil.java:57)
	at org.apache.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:75)
	at org.apache.dubbo.samples.api.GreetingServiceDubboProxy0.sayHello(GreetingServiceDubboProxy0.java)
	at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
	at com.sun.proxy.$Proxy27.sayHello(Unknown Source)
	at org.apache.dubbo.samples.action.GreetingServiceConsumer.doSayHello(GreetingServiceConsumer.java:34)
	at org.apache.dubbo.samples.ConsumerBootstrap.main(ConsumerBootstrap.java:38)
May 31, 2023 5:47:26 AM org.apache.dubbo.rpc.cluster.SingleRouterChain warn
WARNING:  [DUBBO] No provider available after route for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 from registry 10.1.3.76 on the consumer 10.1.3.76 using the dubbo version 3.2.0. Router snapshot is below:
[ Parent (Input: 0) (Current Node Output: 0) (Chain Node Output: 0) ] Input: Empty -> Chain Node Output: Empty
  [ MockInvokersSelector (Input: 0) (Current Node Output: 0) (Chain Node Output: 0) Router message: Empty invokers. Directly return. ] Current Node Output: Empty, dubbo version: 3.2.0, current host: 10.1.3.76, error code: 2-2. This may be caused by No provider available after route for the service, go to https://dubbo.apache.org/faq/2/2 to find instructions.
May 31, 2023 5:47:26 AM org.apache.dubbo.rpc.cluster.directory.AbstractDirectory warn
WARNING:  [DUBBO] No provider available after connectivity filter for the service org.apache.dubbo.samples.api.GreetingService:1.0.0 All validInvokers' size: 0 All routed invokers' size: 0 All invokers' size: 0 from registry istiod.istio-system.svc:15012 on the consumer 10.1.3.76 using the dubbo version 3.2.0., dubbo version: 3.2.0, current host: 10.1.3.76, error code: 2-2. This may be caused by provider server or registry center crashed, go to https://dubbo.apache.org/faq/2/2 to find instructions.

The text was updated successfully, but these errors were encountered:

AYue-94 · 2023-05-31T13:02:26Z

我大概看了一下，主要有两个地方可能有问题。
1）org.apache.dubbo.registry.xds.util.protocol.impl.RdsProtocol
解析的时候没有解析weightcluster。

但是却提供了xdsrouter相关功能，但是实际上底层invokers并不会存在。
2）org.apache.dubbo.rpc.cluster.router.xds.XdsRouter#computeSubset
这里在路由的时候，构造bitlist应该有问题，是不是应该这样做？

除了上面两个问题以外，我想问一下proxyless模式现在是不是非生产可用？

AlbumenJ · 2023-06-02T07:20:38Z

@aamingaa PTAL

aamingaa · 2023-06-02T08:53:53Z

怎么直接apply virtual.yml了，前面还有好几步呢，参考下下面的命令
mvn clean package -DskipTests -U
cd ./dubbo-samples-xds-provider/
docker build -t aamingaa/dubbo-demo:dubbo-samples-xds-provider_0.0.1 .
cd ../dubbo-samples-xds-consumer/
docker build -t aamingaa/dubbo-demo:dubbo-samples-xds-consumer_0.0.1 .
cd ../
kubectl apply -f ./deploy/Namespace.yml
kubens dubbo-demo
cd ./dubbo-samples-xds-provider/src/main/resources/k8s
kubectl apply -f Deployment.yml
kubectl apply -f Deployment-2.yml
kubectl apply -f Service.yml
kubectl apply -f Virtual-Service.yml
cd ../../../../../dubbo-samples-xds-consumer/src/main/resources/k8s
kubectl apply -f Deployment.yml
kubectl apply -f Service.yml

AYue-94 · 2023-06-05T02:12:26Z

怎么直接apply virtual.yml了，前面还有好几步呢，参考下下面的命令 mvn clean package -DskipTests -U cd ./dubbo-samples-xds-provider/ docker build -t aamingaa/dubbo-demo:dubbo-samples-xds-provider_0.0.1 . cd ../dubbo-samples-xds-consumer/ docker build -t aamingaa/dubbo-demo:dubbo-samples-xds-consumer_0.0.1 . cd ../ kubectl apply -f ./deploy/Namespace.yml kubens dubbo-demo cd ./dubbo-samples-xds-provider/src/main/resources/k8s kubectl apply -f Deployment.yml kubectl apply -f Deployment-2.yml kubectl apply -f Service.yml kubectl apply -f Virtual-Service.yml cd ../../../../../dubbo-samples-xds-consumer/src/main/resources/k8s kubectl apply -f Deployment.yml kubectl apply -f Service.yml

1）我省略了前面Deployment和Service的部署，这几步肯定要做的。
2）dubbo-samples-xds并没有提供virtualService.yaml，所以我把sample项目部署、打标、VirtualService资源的定义分开描述了。
3）sidecar模式确实没问题，问题是proxyless模式

aamingaa · 2023-06-05T02:54:53Z

这有一份proxyless下模式的virtualService.yml的配置文件，你对照着试一下？apache/dubbo-samples@master...aamingaa:dubbo-samples:test/xds_debug_3.2.0-beta.7-SNAPSHOT

AYue-94 · 2023-06-06T02:32:17Z

这有一份proxyless下模式的virtualService.yml的配置文件，你对照着试一下？apache/dubbo-samples@master...aamingaa:dubbo-samples:test/xds_debug_3.2.0-beta.7-SNAPSHOT

@aamingaa

这个我也试了，不行。因为sidecar模式和proxyless模式，除了application.properties中几个配置项不一样以外。其他部署上是没有区别的（除了namespace不打标注入istioproxy以外），所以sidecar能通，proxyless我这边也不会部署有什么问题。

如果我没理解错的话，运行时通过PilotExchanger.getInstance().getEndpoints("dubbo-samples-xds-provider")应该能获取到provider的所有Endpoint，包括v1的和v2的，但是现在也获取不到。

songxiaosheng · 2023-09-23T08:58:35Z

这周也遇到了同样的异常ProxyLess模式，

个人mac电脑安装的k8s环境参考如下教程安装的xds通信一切正常，
- https://cn.dubbo.apache.org/zh-cn/blog/2022/09/05/proxyless-mesh%E5%9C%A8dubbo%E4%B8%AD%E7%9A%84%E5%AE%9E%E8%B7%B5/
公司环境遇到了相同的异常(完全使用案例中的Demo)，消费者无法发现提供者，
- 排查发现istio的LDS和RDS日志都有，并且kiali控制台均能看到消费者和提供者应用信息，但是使用3.1.0 3.1.8 3.2.6版本时消费者均无法发现提供者服务，提供者Service也有配置，是否有思路排查来观测消费者查询到的提供者服务

CrazyHZM transferred this issue from apache/dubbo Dec 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dubbo-xds not support WeightedCluster #271

dubbo-xds not support WeightedCluster #271

AYue-94 commented May 31, 2023 •

edited

Loading

AYue-94 commented May 31, 2023 •

edited

Loading

AlbumenJ commented Jun 2, 2023

aamingaa commented Jun 2, 2023

AYue-94 commented Jun 5, 2023 •

edited

Loading

aamingaa commented Jun 5, 2023

AYue-94 commented Jun 6, 2023 •

edited

Loading

songxiaosheng commented Sep 23, 2023 •

edited

Loading

dubbo-xds not support WeightedCluster #271

dubbo-xds not support WeightedCluster #271

Comments

AYue-94 commented May 31, 2023 • edited Loading

Environment

Steps to reproduce this issue

Expected Behavior

Actual Behavior

AYue-94 commented May 31, 2023 • edited Loading

AlbumenJ commented Jun 2, 2023

aamingaa commented Jun 2, 2023

AYue-94 commented Jun 5, 2023 • edited Loading

aamingaa commented Jun 5, 2023

AYue-94 commented Jun 6, 2023 • edited Loading

songxiaosheng commented Sep 23, 2023 • edited Loading

AYue-94 commented May 31, 2023 •

edited

Loading

AYue-94 commented May 31, 2023 •

edited

Loading

AYue-94 commented Jun 5, 2023 •

edited

Loading

AYue-94 commented Jun 6, 2023 •

edited

Loading

songxiaosheng commented Sep 23, 2023 •

edited

Loading