Skip to content

Latest commit

 

History

History
90 lines (59 loc) · 7.85 KB

sidecar-protocol.md

File metadata and controls

90 lines (59 loc) · 7.85 KB
Raw

Background

  • EaseMesh uses Easegress-based sidecar inside Kubernetes Pod for traffic hosting and EaseAgent for metrics reporting and RESTful-API-based RPC enhancement.
  • EaseMesh only supports Java SpringCloud ecosystem's application natively currently.

EaseMesh traffic hosting

There are three types of traffic that are managed by EaseMesh.

  • First, the RESTful-API HTTP traffic for RPC inside the mesh. This traffic is invoked by Java applications with popular RPC frameworks, such as Feign, RestTemplate, and so on. EaseAgent will enhance this traffic by adding the target RPC server's name inside the HTTP header for telling the sidecar of the real handler. The traffic must satisfy at least one way of:
  1. Headers: X-Mesh-Rpc-Service: {destination_service_name}
  2. Headers: Host: {destination_service_name} or Host: ^(\w+\.)*{destination_service_name}\.(\w+)\.svc\..+
  • Second, the Health-checking HTTP traffic. This traffic is sent from the sidecar to the Java application's additional port opened by EaseAgent. The complete URI is http://localhost:9900/health by default. This 9900 port is opened by EaseAgent, sidecar will query this URI period for checking the liveness of the Java application. After successfully deployed, sidecar will registry this instance into EaseMesh automatically after confirming the HTTP 200 success return by this URI.
  • Third, the Service-discovery traffic. This traffic is invoked by the Java spring cloud application's RPC framework. During the lifetime of the Java application, sidecar will work as the Java application's service registry and discovery center. EaseMesh sidecar implements Eureka/Consul/Naocs APIs for hosting the Java application's registry and discovery requests. To make the sidecar server the registry and discovery center, value it with http://localhost:13009 inside the Java application's XML. The port 13009 is listened by sidecar for handling Eureka/Consul/Nacos APIs.

The ports used by EaseMesh sidecar+agent system

Role Port Description
Sidecar 13001 The default Ingress port listened by sidecar for handing over traffic to local Java application
Sidecar 13002 The default egress port listened by sidecar for routing local Java applications RPC request to another Java application
Sidecar 13009 The default registry and discovery port listened by sidecar, for handling local Java application's Eureka/Conslu/Nacos APIs
Agent 9900 The default health port listened by Agent queried by sidecar for checking the liveness of Java application
Application customized port The port listened by the user application. The sidecar routes ingress traffic to it

Problem

  • Figuring out the standard for supporting multiple-language programs running inside EaseMesh.

Analysis

  • To support the none-Java-spring-cloud-based RESTful-API application, we had demoed a DNS-enhancement way for supporting Java spring boot application. Can we reuse this way to support Golang-based or RUST-based RESTful-API applications?

  • To support non-registry-discovery dependent on Java spring boot application, EaseMesh enhances Kubernetes' coreDNS with add a plugin for finding services inside EaseMesh's Etcd. We can reuse this method for none-Java-based programs.
  • EaseAgent uses Java Byte Buddy-based technology for collecting several application metrics. This requires a JVM-liked software architecture. This observability will be sacrificed for the none-Java-spring-cloud-based RESTful-API application.

Protocol

To support the none-Java-spring-cloud-based RESTful-API application, regardless of which programming is used. The application must follow the protocol below

  1. It must serve as standard RESTful-API for handling requesting or invoking RPC.

  2. It must use a domain for discovering in RESTful-API RPC.

Requirement:
1. Use coreDNS with easemesh specific plugin
2. Valid domain formats:
  - Service name, e.g. `vet-services`
  - Service name with more subdomains, e.g.
    - `_tcp.vet-services.easemesh.svc.cluster.local`
    - `vet-services.easemesh.svc.cluster.local`
    - `_zip._tcp.vet-services.easemesh.svc.com`

  1. It must serve the http://localhost:9900/health URI for EaseMesh health checking. (Only HTTP 200 return is required, regardless of the body content)

  2. It must reserve ports 13001 , 13002 and 13009 for local sidecar usage.

  3. It should specify the application port in Kubernetes deployment's mesh.megaease.com/application-port annotation for sidecar routing the ingress traffic. If it is omitted, the first port of the first container will be regarded as the application port.

  4. The sidecar periodically post agent config to http://localhost:9900/config to notify the latest config. The config body is like(some fields are ommitted)

{
  "easeagent.progress.forwarded.headers": "X-Location,X-Mesh-Service-Canary,X-Phone-Os",
  "loadBalance.policy": "random",
  "name": "service-001",
  "registerTenant": "mesh-tenant",
  "sidecar.address": "127.0.0.1",
  "sidecar.discoveryType": "consul",
  "sidecar.egressPort": "13002",
  "sidecar.egressProtocol": "http",
  "sidecar.ingressPort": "13001",
  "sidecar.ingressProtocol": "http"
}

For example, we extract the value of easeagent.progress.forwarded.headers to get the canary headers needed to transmit across the chain.

If an application obeys the protocol above, then EaseMesh can run it inside with sacrificed observability regardless of the implements programming language.