Support available layers of service in the topology.

docs/en/changes/changes.md

@@ -38,13 +38,14 @@
   - VIRTUAL_DATABASE -> MYSQL
   - VIRTUAL_DATABASE -> POSTGRESQL
 * Add Golang as a supported language for AMQP.
+* Support available layers of service in the topology.
 
 #### UI
 
 * Fix the mismatch between the unit and calculation of the "Network Bandwidth Usage" widget in Linux-Service Dashboard.
 * Add theme change animation.
 * Implement the Service and Instance hierarchy topology.
-* Support Tabs in the widget visiable when MQE expressions.
+* Support Tabs in the widget visible when MQE expressions.
 * Support search on Marketplace.
 * Fix default route.
 * Fix layout on the Log widget.
@@ -55,6 +56,10 @@
 * Fix dashboard `K8S-Service-Root` metrics expression.
 * Add dashboards for Service/Instance Hierarchy.
 * Fix MQE in dashboards when using `Card widget`.
+* Optimize tooltips style.
+* Fix resizing window causes the trace graph to display incorrectly.
+* Add the not found page(404).
+* Enhance VNode logic and support multiple Trace IDs in span's ref.
 
 #### Documentation
 

docs/en/swip/SWIP-4.md

@@ -0,0 +1,57 @@
+# Support available layers of service in the topology.
+
+## Motivation
+
+UI could jump to the service dashboard and query service hierarchy from the topology node. 
+For now topology node includes name and ID but without layer, as the service could have multiple layers,
+the limitation is that it is only works on the current layer which the topology represents:
+1. UI could not jump into another layer's dashboard of the service.
+2. UI could not query the service hierarchy from the topology node if the node is not in current layer.
+
+Here are typical use cases:
+should have a chance to jump into another layer's dashboard of the service:
+1. In the mesh topology, mesh(layer MESH) and mesh-dp(layer MESH_DP) share a similar topology, one node will have two layers.
+2. In the mesh topology, agent(layer GENERAL) + virtual database(layer VIRTUAL_DATABASE), the node is in different layers.
+
+Both of these two cases have hybrid layer topology. If we could support that, we could have a better x-layer interaction.
+
+## Architecture Graph
+
+There is no significant architecture-level change.
+
+## Propose Changes
+
+Add the layers info into topology node:
+1. When building the topology node fetch the layers info from the service according to the service id.
+2. Return `layers` info in the `Node` when query the topology.
+
+## Imported Dependencies libs and their licenses.
+
+No new library is planned to be added to the codebase.
+
+## Compatibility
+
+About the **protocol**, there should be no breaking changes, but enhancements only. New field `layers` is going to be added to the
+`Node` in the query protocol `topology.graphqls`.
+
+```graphql
+type Node {
+  # The service ID of the node.
+  id: ID!
+  # The literal name of the #id.
+  name: String!
+  # The type name may be
+  # 1. The service provider/middleware tech, such as: Tomcat, SpringMVC
+  # 2. Conjectural Service, e.g. MySQL, Redis, Kafka
+  type: String
+  # It is a conjecture node or real node, to represent a service or endpoint.
+  isReal: Boolean!
+  # The layers of the service.
+  layers: [String!]!
+}
+```
+
+## General usage docs
+
+This proposal doesn't impact the end user in any way of using SkyWalking. The remarkable change will be in the UI topology map, 
+users could jump into the proper layer's service dashboard and query the service hierarchy from the topology node.

docs/en/swip/readme.md

@@ -71,6 +71,7 @@ All accepted and proposed SWIPs could be found in [here](https://github.com/apac
 Next SWIP Number: 4
 
 ### Accepted SWIPs
+- [SWIP-4 Support available layers of service in the topology](SWIP-4.md)
 - [SWIP-3 Support RocketMQ Monitoring](SWIP-3.md)
 - [SWIP-2 Collecting and Gathering Kubernetes Monitoring Data](SWIP-2.md)
 - [SWIP-1 Create and detect Service Hierarchy Relationship](SWIP-1.md)

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilder.java

@@ -22,6 +22,7 @@
 import java.util.LinkedList;
 import java.util.List;
 import java.util.Map;
+import lombok.SneakyThrows;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.skywalking.oap.server.core.Const;
 import org.apache.skywalking.oap.server.core.CoreModule;
@@ -31,6 +32,7 @@
 import org.apache.skywalking.oap.server.core.config.IComponentLibraryCatalogService;
 import org.apache.skywalking.oap.server.core.query.type.Call;
 import org.apache.skywalking.oap.server.core.query.type.Node;
+import org.apache.skywalking.oap.server.core.query.type.Service;
 import org.apache.skywalking.oap.server.core.query.type.Topology;
 import org.apache.skywalking.oap.server.core.source.DetectPoint;
 import org.apache.skywalking.oap.server.library.module.ModuleManager;
@@ -41,6 +43,8 @@ class ServiceTopologyBuilder {
     private final IComponentLibraryCatalogService componentLibraryCatalogService;
     private final NetworkAddressAliasCache networkAddressAliasCache;
     private final String userID;
+    private final ModuleManager moduleManager;
+    private MetadataQueryService metadataQueryService;
 
     ServiceTopologyBuilder(ModuleManager moduleManager) {
         this.componentLibraryCatalogService = moduleManager.find(CoreModule.NAME)
@@ -50,6 +54,16 @@ class ServiceTopologyBuilder {
                                                      .provider()
                                                      .getService(NetworkAddressAliasCache.class);
         this.userID = IDManager.ServiceID.buildId(Const.USER_SERVICE_NAME, false);
+        this.moduleManager = moduleManager;
+    }
+
+    private MetadataQueryService getMetadataQueryService() {
+        if (metadataQueryService == null) {
+            this.metadataQueryService = moduleManager.find(CoreModule.NAME)
+                                                     .provider()
+                                                     .getService(MetadataQueryService.class);
+        }
+        return metadataQueryService;
     }
 
     Topology build(List<Call.CallDetail> serviceRelationClientCalls, List<Call.CallDetail> serviceRelationServerCalls) {
@@ -193,11 +207,16 @@ Topology build(List<Call.CallDetail> serviceRelationClientCalls, List<Call.CallD
         return topology;
     }
 
+    @SneakyThrows
     private Node buildNode(String sourceId, IDManager.ServiceID.ServiceIDDefinition sourceService) {
         Node serviceNode = new Node();
         serviceNode.setId(sourceId);
         serviceNode.setName(sourceService.getName());
         serviceNode.setReal(sourceService.isReal());
+        Service service = getMetadataQueryService().getService(sourceId);
+        if (service != null) {
+            serviceNode.getLayers().addAll(service.getLayers());
+        }
         return serviceNode;
     }
 }

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/query/type/Node.java

@@ -18,6 +18,8 @@
 
 package org.apache.skywalking.oap.server.core.query.type;
 
+import java.util.HashSet;
+import java.util.Set;
 import lombok.Getter;
 import lombok.Setter;
 import org.apache.skywalking.oap.server.core.query.NotGraphQLField;
@@ -35,6 +37,9 @@ public class Node {
     @Getter
     @Setter
     private boolean isReal;
+    @Getter
+    @Setter
+    private Set<String> layers = new HashSet<>();
 
     /**
      * A flag indicate whether the {@link #type} has been set from the call detected from service side.

oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/Hierarchy/HierarchyQueryServiceTest.java → oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/hierarchy/HierarchyQueryServiceTest.java

@@ -16,7 +16,7 @@
  *
  */
 
-package org.apache.skywalking.oap.server.core.Hierarchy;
+package org.apache.skywalking.oap.server.core.hierarchy;
 
 import java.util.HashMap;
 import java.util.Map;

oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/query/ServiceTopologyBuilderTest.java

@@ -20,32 +20,41 @@
 
 import java.util.ArrayList;
 import java.util.List;
+import java.util.Set;
+import lombok.SneakyThrows;
 import org.apache.skywalking.oap.server.core.CoreModule;
 import org.apache.skywalking.oap.server.core.CoreModuleConfig;
 import org.apache.skywalking.oap.server.core.CoreModuleProvider;
 import org.apache.skywalking.oap.server.core.MockModuleManager;
 import org.apache.skywalking.oap.server.core.MockModuleProvider;
 import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.Layer;
 import org.apache.skywalking.oap.server.core.cache.NetworkAddressAliasCache;
 import org.apache.skywalking.oap.server.core.config.ComponentLibraryCatalogService;
 import org.apache.skywalking.oap.server.core.config.IComponentLibraryCatalogService;
 import org.apache.skywalking.oap.server.core.query.type.Call;
 import org.apache.skywalking.oap.server.core.query.type.Node;
+import org.apache.skywalking.oap.server.core.query.type.Service;
 import org.apache.skywalking.oap.server.core.query.type.Topology;
 import org.apache.skywalking.oap.server.core.source.DetectPoint;
 import org.apache.skywalking.oap.server.library.module.ModuleManager;
 import org.junit.jupiter.api.Assertions;
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.mockito.Mockito;
+import org.powermock.reflect.Whitebox;
+
+import static org.mockito.Mockito.when;
 
 public class ServiceTopologyBuilderTest {
     private CoreModuleProvider moduleProvider;
     private ModuleManager moduleManager;
+    private MetadataQueryService metadataQueryService;
 
     @BeforeEach
     public void setupMetrics() throws Throwable {
         moduleProvider = Mockito.mock(CoreModuleProvider.class);
+        metadataQueryService = Mockito.mock(MetadataQueryService.class);
         moduleManager = new MockModuleManager() {
             @Override
             protected void init() {
@@ -62,16 +71,22 @@ protected void register() {
         };
     }
 
+    @SneakyThrows
     @Test
     public void testServiceTopologyBuild() {
+        Service svrA = getSvrA();
+        Service svrB = getSvrB();
         final ServiceTopologyBuilder serviceTopologyBuilder = new ServiceTopologyBuilder(moduleManager);
+        Whitebox.setInternalState(serviceTopologyBuilder, "metadataQueryService", metadataQueryService);
+        when(metadataQueryService.getService(svrA.getId())).thenReturn(svrA);
+        when(metadataQueryService.getService(svrB.getId())).thenReturn(svrB);
         List<Call.CallDetail> serviceRelationClientCalls = new ArrayList<>();
         Call.CallDetail call1 = new Call.CallDetail();
         call1.buildFromServiceRelation(
             IDManager.ServiceID.buildRelationId(
                 new IDManager.ServiceID.ServiceRelationDefine(
-                    IDManager.ServiceID.buildId("SvrA", true),
-                    IDManager.ServiceID.buildId("SvrB", true)
+                    IDManager.ServiceID.buildId(svrA.getName(), true),
+                    IDManager.ServiceID.buildId(svrB.getName(), true)
                 )
             ),
             // mtls
@@ -83,8 +98,8 @@ public void testServiceTopologyBuild() {
         call2.buildFromServiceRelation(
             IDManager.ServiceID.buildRelationId(
                 new IDManager.ServiceID.ServiceRelationDefine(
-                    IDManager.ServiceID.buildId("SvrA", true),
-                    IDManager.ServiceID.buildId("SvrB", true)
+                    IDManager.ServiceID.buildId(svrA.getName(), true),
+                    IDManager.ServiceID.buildId(svrB.getName(), true)
                 )
             ),
             // http
@@ -98,8 +113,8 @@ public void testServiceTopologyBuild() {
         call3.buildFromServiceRelation(
             IDManager.ServiceID.buildRelationId(
                 new IDManager.ServiceID.ServiceRelationDefine(
-                    IDManager.ServiceID.buildId("SvrA", true),
-                    IDManager.ServiceID.buildId("SvrB", true)
+                    IDManager.ServiceID.buildId(svrA.getName(), true),
+                    IDManager.ServiceID.buildId(svrB.getName(), true)
                 )
             ),
             // mtls
@@ -111,8 +126,8 @@ public void testServiceTopologyBuild() {
         call4.buildFromServiceRelation(
             IDManager.ServiceID.buildRelationId(
                 new IDManager.ServiceID.ServiceRelationDefine(
-                    IDManager.ServiceID.buildId("SvrA", true),
-                    IDManager.ServiceID.buildId("SvrB", true)
+                    IDManager.ServiceID.buildId(svrA.getName(), true),
+                    IDManager.ServiceID.buildId(svrB.getName(), true)
                 )
             ),
             // http
@@ -125,13 +140,33 @@ public void testServiceTopologyBuild() {
         for (final Node node : topology.getNodes()) {
             if (node.getName().equals("SvrB")) {
                 Assertions.assertEquals("http", node.getType());
+                Assertions.assertEquals(Set.of(Layer.MESH.name(), Layer.MESH_DP.name()), node.getLayers());
             } else if (node.getName().equals("SvrA")) {
                 Assertions.assertEquals(null, node.getType());
+                Assertions.assertEquals(Set.of(Layer.GENERAL.name()), node.getLayers());
             }
         }
         for (final Call call : topology.getCalls()) {
             Assertions.assertEquals(2, call.getSourceComponents().size());
             Assertions.assertEquals(List.of("mtls", "http"), call.getTargetComponents());
         }
     }
+
+    private Service getSvrA() {
+        Service service = new Service();
+        service.setId(IDManager.ServiceID.buildId("SvrA", true));
+        service.setName("SvrA");
+        service.setShortName("SvrA");
+        service.setLayers(Set.of(Layer.GENERAL.name()));
+        return service;
+    }
+
+    private Service getSvrB() {
+        Service service = new Service();
+        service.setId(IDManager.ServiceID.buildId("SvrB", true));
+        service.setName("SvrB");
+        service.setShortName("SvrB");
+        service.setLayers(Set.of(Layer.MESH.name(), Layer.MESH_DP.name()));
+        return service;
+    }
 }