Skip to content
Permalink
Browse files

SCB-1478 support API gateway (#73)

  • Loading branch information...
tianxiaoliang committed Sep 11, 2019
1 parent f920375 commit 4839e488091c6865ae17d354be6a5f2a85f881fb
Showing with 910 additions and 295 deletions.
  1. +2 −3 README.md
  2. +4 −3 cmd/mesher/mesher.go
  3. +1 −0 docs/configuration.rst
  4. +1 −1 docs/configurations/cli.md
  5. +78 −0 docs/configurations/edge.md
  6. +20 −0 docs/development.rst
  7. +7 −0 docs/development/build.md
  8. +33 −0 docs/development/cloud-provider.md
  9. +42 −0 docs/development/handler-chain.md
  10. +1 −0 docs/index.rst
  11. +78 −0 examples/edge/conf/chassis.yaml
  12. +52 −0 examples/edge/conf/mesher.yaml
  13. +3 −0 examples/edge/conf/microservice.yaml
  14. +6 −5 go.mod
  15. +6 −5 proxy/bootstrap/bootstrap.go
  16. +5 −5 proxy/cmd/cmd.go
  17. +6 −6 proxy/common/common.go
  18. +14 −4 proxy/config/config.go
  19. +1 −0 proxy/config/struct.go
  20. +77 −0 proxy/config/struct_ingress.go
  21. +62 −0 proxy/config/struct_ingress_test.go
  22. +32 −41 proxy/{util/compare.go → ingress/ingress.go}
  23. +78 −0 proxy/ingress/servicecomb/ingress.go
  24. +72 −0 proxy/ingress/servicecomb/listener.go
  25. +62 −14 proxy/pkg/egress/egress_test.go
  26. +2 −2 proxy/pkg/runtime/runtime.go
  27. +4 −4 proxy/protocol/dubbo/proxy/dubbo_proxy_ouput.go
  28. +2 −2 proxy/protocol/grpc/reverse_proxy.go
  29. +3 −3 proxy/protocol/grpc/server.go
  30. +107 −0 proxy/protocol/http/gateway.go
  31. +21 −16 proxy/protocol/http/http_server.go
  32. +3 −3 proxy/protocol/http/{reverse_proxy.go → sidecar.go}
  33. +3 −4 proxy/resolver/authority/destination.go
  34. +7 −10 proxy/resolver/authority/destination_test.go
  35. +7 −7 proxy/resolver/destination.go
  36. +8 −10 proxy/resolver/destination_test.go
  37. +0 −147 proxy/util/compara_test.go
@@ -14,11 +14,11 @@ you can develop micro services with java chassis or go chassis frameworks
and use mesher to make other service join to the same system.
- flexible: you can develop and customize your own service mesh
- OS: support both linux and windows OS, which means you can govern your services writen in .net with java, go etc.

- API gateway: mesher is able to run as a independent edge service and manage ingress traffic.
# Features
- Build on top of go micro service framework: so that mesher has all of features of
[go chassis](https://github.com/go-chassis/go-chassis),a high flexible go micro service framework.
you can custom your own service mesh by extending lots of components.
you can custom your own service mesh and API gateway by extending lots of components.
- Admin API:Listen on an isolated port, expose useful runtime information and metrics.
- support protocols: http and grpc
- No IP tables forwarding: Mesher leverage
@@ -58,7 +58,6 @@ it will build binary and docker image
- tar file: release/mesher-latest-linux-amd64.tar
- docker image name: servicecomb/mesher-sidecar:latest

# Documentations
# Documentations
You can see more documentations in [here](https://mesher.readthedocs.io/en/latest/),
this online doc is for latest version of mesher, if you want to see your version's doc,
@@ -26,12 +26,13 @@ import (
_ "github.com/apache/servicecomb-mesher/proxy/protocol/dubbo/client/chassis"
_ "github.com/apache/servicecomb-mesher/proxy/protocol/dubbo/server"
_ "github.com/apache/servicecomb-mesher/proxy/protocol/dubbo/simpleRegistry"

_ "github.com/go-chassis/go-chassis/configcenter" //use config center
// config server
_ "github.com/go-chassis/go-chassis-config/servicecombkie"
//protocols
_ "github.com/apache/servicecomb-mesher/proxy/protocol/grpc"
_ "github.com/apache/servicecomb-mesher/proxy/protocol/http"

//ingress rule fetcher
_ "github.com/apache/servicecomb-mesher/proxy/ingress/servicecomb"
"github.com/apache/servicecomb-mesher/proxy/server"

_ "github.com/apache/servicecomb-mesher/proxy/pkg/egress/archaius"
@@ -10,3 +10,4 @@ User guides
configurations/admin
configurations/health
configurations/destination_resolver
configurations/edge
@@ -13,7 +13,7 @@ mesher --config=mesher.yaml --service-ports=rest:8080

**--mode**
>*(optional, string)* mesher has 2 work mode, sidecar and per-host, default is sidecar
>*(optional, string)* mesher has 2 work mode, sidecar and edge, default is sidecar

**--service-ports**
@@ -0,0 +1,78 @@
# API gateway
mesher is able to work as a API gateway to mange traffic,to run mesher as an API gateway
```shell
mesher --config=mesher.yaml --mode edge
```
the ingress rule is in mesher.yaml

### Options

**mesher.ingress.type**
>*(optional, string)* default is servicecomb, it reads servicecomb ingress rule.
>it is a plugin, you can custom your own implementation

**mesher.ingress.rule.http**
>*(optional, string)* rule about how to forward http traffic. it holds a yaml content as rule.
below explain the content, the rule list is like a filter, all the request will go through this rule list until match one rule.

**apiPath**
>*(required, string)* if request's url match this, it will use this rule
**host**
>*(optional, string)* if request HOST match this, mesher will use this rule, it can be empty,
>if you set both host and apiPath, the request's host and api path must match them at the same time
>
**service.name**
>*(required, string)* target backend service name in registry service(like ServiceComb service center)
>
**service.redirectPath**
>*(optional, string)* by default, mesher use original request's url
>
**service.port.value**
>*(optional, string)* if you use java chassis or go chassis to develop backend service, no need to set it.
>but if your backend service use mesher-sidecar, you must give your service port here.
>
### example
```yaml
mesher:
ingress:
type: servicecomb
rule:
http: |
- host: example.com
apiPath: /some/api
service:
name: example
redirectPath: /another/api
port:
name: http-legacy
value: 8080
- apiPath: /some/api
service:
name: Server
port:
name: http
value: 8080
```

### Enable TLS
generate private key
```sh
openssl genrsa -out server.key 2048
```
sign cert with private key
```shell script
openssl req -new -x509 -key server.key -out server.crt -days 3650
```
set file path in chassis.yaml
```yaml
ssl:
mesher-edge.rest.Provider.certFile: server.crt
mesher-edge.rest.Provider.keyFile: server.key
```

To know advanced feature about TLS configuration, check
https://docs.go-chassis.com/user-guides/tls.html
@@ -0,0 +1,20 @@
Development guides
=========================
mesher is an out of box service mesh and API gateway component,
you can use them by simply setting configuration files.
But some of user still need to customize a service mesh or API gateway.
For example:

- API gateway need to query account system and do the authentication and authorization.
- mesher need to access cloud provider API
- mesher use customized control panel
- mesher use customized config server


.. toctree::
:maxdepth: 4
:glob:

development/handler-chain
development/cloud-provider
development/build
@@ -0,0 +1,7 @@
# Build mesher
you need to build and release your mesher after the customization

### Build binary
you can refer to build/build_proxy to see how we build mesher binary and docker image.

build/docker/proxy/Dockerfile is a example about how to make a docker image
@@ -0,0 +1,33 @@
# Cloud Provider
By default Mesher do not support any cloud provider.
But there is plugin that helps mesher to do it.

### Huawei Cloud
Mesher is able to use huawei cloud ServiceComb engine.
#### Access ServiceComb Engine API
import auth in cmd/mesher/mesher.go
```go
import _ "github.com/huaweicse/auth/adaptor/gochassis"
```

it will sign all requests between mesher to ServiceComb Engine.

#### Use Config Center to manage configuration
Mesher use servicecomb-kie as config server,
```go
_ "github.com/go-chassis/go-chassis-config/servicecombkie"
```
when you need to use ServiceComb Engine, you must replace this line.
import config center in cmd/mesher/mesher.go.
```go
_ "github.com/go-chassis/go-chassis-config/configcenter"
```
set the config center in chassis.yaml
```yaml
config:
client:
serverUri: https://xxx #endpoint of servicecomb engine
refreshMode: 1 # 1: only pull config.
refreshInterval: 30 # unit is second
type: config_center
```
@@ -0,0 +1,42 @@
# Handler chain
all the traffic will go through the handler chain.
A chain is composite of handlers, each handler has a particular logic.
Mesher also has a lots of feature working in chain, like route management, circuit breaking, rate-limiting.
In Summary, handler is the middle ware between client and servers,
it is useful, when you want to add authorization to intercept illegal requests.

### How to write a handler
https://docs.go-chassis.com/dev-guides/how-to-implement-handler.html

### How to use it in handler chain
in chassis.yaml add your handler name in chain configuration.
as side car and API gateway, mesher's chain has different meaning.

For example, running as mesher-sidecar, service A call another service B,
outgoing chain process all the service A requests before remote call,
incoming chain process all the requests from service A, before access to service B API.

In summary outgoing chain works when a service attempt to call other services,
incoming chain works when other services call this service
```yaml
handler:
chain:
Consumer:
# if a service call other service, it go through this chain, loadbalance and transport is must
outgoing: router, bizkeeper-consumer, loadbalance, transport
Provider:
incoming: ratelimiter-provider
```

running as API gateway,
incoming chain process all the requests from the external network,
outgoing chain process all the the request between API gateway and backend services
```yaml
handler:
chain:
Consumer:
#loadbalance and transport is must
outgoing: router, bizkeeper-consumer, loadbalance, transport
Provider:
incoming: ratelimiter-provider
```
@@ -13,6 +13,7 @@ Welcome to Mesher's documentation!
intro
get-started
configuration
development
protocols
istio-guides
sidecar
@@ -0,0 +1,78 @@
---
cse:
protocols:
http:
listenAddress: 127.0.0.1:30101
rest-admin:
listenAddress: 127.0.0.1:30102 # listen addr use to adminAPI
service:
registry:
address: http://127.0.0.1:30100 # uri of service center
#address: https://cse.cn-north-1.myhuaweicloud.com:443 # uri of service center
scope: full #set full to be able to discover other app's service
watch: false # set if you want to watch instance change event
autoIPIndex: true # set to true if u want to resolve source IP to micro service
# config:
# client:
# serverUri: https://127.0.0.1:30110 #uri of config center
# type: servicecomb-kie
# refreshMode: 1 # 1: only pull config.
# refreshInterval: 30 # unit is second
# monitor: #Send monitoring data to CSE monitor Server
# client:
# serverUri: https://cse.cn-north-1.myhuaweicloud.com:443 # monitor server url
handler:
chain:
Consumer:
outgoing: router,bizkeeper-consumer,loadbalance,tracing-consumer,transport #consumer handlers
Provider:
incoming: tracing-provider #provider handlers
# loadbalance:
# strategy:
# name: RoundRobin # Random|RoundRobin|SessionStickiness
# retryEnabled: false # if there is error, retry request or not
# retryOnNext: 2 # times to switch to another instance based on strategy
# retryOnSame: 3 # times to retry on the same instance
# backoff: # backoff policy of retried request
# kind: constant # jittered/constant/zero
# MinMs: 200 # millisecond, Minimum duration to backoff
# MaxMs: 400 # millisecond, Maximum duration to backoff
## circuit breaker configurations
# isolation:
# Consumer:
# timeout:
# enabled: true
# timeoutInMilliseconds: 1000
# maxConcurrentRequests: 100
# circuitBreaker:
# Consumer:
# enabled: true
# forceOpen: false
# forceClosed: false
# sleepWindowInMilliseconds: 10000
# requestVolumeThreshold: 20
# errorThresholdPercentage: 50
# fallback:
# Consumer:
# enabled: true
# maxConcurrentRequests: 20
# fallbackpolicy:
# Consumer:
# policy: throwexception

## Mesher TLS is base on Go Chassis TLS config
ssl:
# mesher-edge.rest.Provider.cipherPlugin: default
# mesher-edge.rest.Provider.verifyPeer: false
# mesher-edge.rest.Provider.cipherSuits: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
# mesher-edge.rest.Provider.protocol: TLSv1.2
# mesher-edge.rest.Provider.caFile:
mesher-edge.rest.Provider.certFile: server.crt
mesher-edge.rest.Provider.keyFile: server.key
# mesher-edge.rest.Provider.certPwdFile:

#tracing:
# enabled: true #enable distribution tracing
# collectorType: zipkin #zipkin: Send tracing info to zipkin server
# #namedPipe: Write tracing info to linux named pipe.
# collectorTarget: http://localhost:9411/api/v1/spans #If the collectorType is "zipkin", the target is a zipkin server url, if the collecterType is "file" or "namedPipe", the target is a file path.
@@ -0,0 +1,52 @@
## Router rules and fault injection rules are moved to router.yaml
#plugin:
# destinationResolver:
# http: host # how to turn host to destination name. default to service name,

admin: #admin API
goRuntimeMetrics : true # enable metrics
enable: true

## enable pprof to profile mesher runtime
#pprof:
# enable: false


# this health check will ping local service port to check if service is still alive, if service can not reachable, mesher
# will update status to OUT_OF_SERVICE in service center
#localHealthCheck:
# - port: 8080
# uri: /health
# interval: 30s
# match:
# status: 200
# body: ok


mesher:
ingress:
type: servicecomb
rule:
http: |
- host: example.com
limit: 30
apiPath: /some/api
service:
name: example
redirectPath: /another/api
port:
name: http-legacy
value: 8080
- apiPath: /sayerror/api
service:
name: Server
redirectPath: /sayerror
port:
name: http
value: 8080
- apiPath: /some/api
service:
name: Server
port:
name: http
value: 8080
@@ -0,0 +1,3 @@
service_description:
name: mesher-edge
version: 0.0.1

0 comments on commit 4839e48

Please sign in to comment.
You can’t perform that action at this time.