Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement "rewrite" using Wasm Plugin #339

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
120 changes: 120 additions & 0 deletions plugins/wasm-go/extensions/rewrite/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# 功能说明

`rewrite` 插件可以用于修改请求域名(Host)以及请求路径(Path),通常用于后端服务的域名/路由与网关侧域名/路由不一致时的配置,与 [Rewrite Annotation](https://higress.io/zh-cn/docs/user/annotation-use-case/#rewrite%E9%87%8D%E5%86%99path%E5%92%8Chost) 实现的效果一致。


# 配置字段

| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|---------------|-----------------|------|-----|------------------------------|
| rewrite_rules | array of object | 选填 | - | 配置请求域名(Host)与请求路径(Path)的重写规则 |

`rewrite_rules` 中配置字段说明如下:

| 名称 | 数据类型 | 填写要求 | 默认值 | 描述 |
|-----------------|-----------------|------|-------|-------------------------------------------------------------------------------|
| match_path_type | string | 选填 | - | 配置请求路径的匹配类型,可选值为:前缀匹配 prefix, 精确匹配 exact, 正则匹配 regex |
| case_sensitive | bool | 选填 | false | 配置匹配时是否区分大小写,默认不区分 |
| match_hosts | array of string | 选填 | - | 配置会被重写的请求域名列表,支持精确匹配(hello.world.com),最左通配(\*.world.com)和最右通配(hello.world.\*) |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match_hosts 和 match_paths 的功能跟插件机制自带的匹配功能重复了,这个插件应当只处理匹配后的重写机制。
此外,路径重写需要考虑正则重写(看到已经支持)和前缀重写。目前重写 annotation 的设计是根据当前路径匹配类型自动设定重写类型,例如精确匹配->精确重写;前缀匹配->前缀重写。
这个插件可以提供用户更灵活的选择,让用户自行选择重写机制。对于前缀和正则重写,需要设定一个重写目标,字段名可以是 rewrite_target。

Copy link
Collaborator Author

@WeixinX WeixinX May 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@johnlanni 感谢指出问题 ~

那么这个插件的模型是否可以改写为:

type RewriteConfig struct {
	rewriteRules []RewriteRule
}

type RewriteRule struct {
	caseSensitive bool   // 大小写是否敏感
	rewriteType   string // 请求路径重写类型 prefix | exact | regex
	rewriteHost   string // 请求域名会被重写为 rewriteHost
	rewritePath   string // 请求路径会被重写为 rewritePath
}

根据文档中 rewrite annotation 的使用示例,我觉得 rewrite_target 和这里的 rewrite_path 功能一样,但在源码中同时有 rewrite-path 和 -target 两个 annotation,我想请问它们的区别是什么?🙏🏻

Copy link
Collaborator

@johnlanni johnlanni May 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我上面说的是 rewrite-path 的机制;
rewrite-target 是为了兼容 nginx ingress 注解设计的,可以看下 nginx ingress的文档:https://kubernetes.github.io/ingress-nginx/examples/rewrite/

这个配置可以的,不过对于正则重写,还需有个配置用于正则匹配捕获,来实现上面 rewrite target 的能力

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

还有一个问题想讨教 ~
前缀重写指的是例如:
请求路径 /v1/api/get 匹配到了路由 /v1/api,此时重写类型为 prefix,重写路径为 /v2,因此该请求路径被重写为 /v2/get 吗?

Copy link
Collaborator

@johnlanni johnlanni May 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type RewritePath struct {
     type string // prefix | exact | regex
     matchPath string // 用于正则匹配并捕获,或者前缀匹配
     caseSensitive bool
     rewritePath string
}
type RewriteRule struct {
	rewriteHost   string 
	rewritePath   RewritePath
}

这样可能更好,作用字段含义更明确些

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

还有一个问题想讨教 ~ 前缀重写指的是例如: 请求路径 /v1/api/get 匹配到了路由 /v1/api,此时重写类型为 prefix,重写路径为 /v2,因此该请求路径被重写为 /v2/get 吗?

参考上面配置格式,如果 type是 prefix,matchPath是 /v1/api,rewritePath是/v2,那么会被重写为 /v2/get

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感谢 ~ 我有空重构一下。另外是否需要添加 e2e usecase?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

嗯,目前做插件的 e2e 需要提供现成的 oci 镜像,这个我记个 issue ,需要优化下 e2e 流程来支持

| match_paths | array of string | 选填 | - | 配置会被重写的请求路径列表,支持 [RE2](https://pkg.go.dev/regexp/syntax) 正则表达式语法 |
| rewrite_host | string | 选填 | - | 配置重写的目标域名 |
| rewrite_path | string | 选填 | - | 配置重写的目标路径 |

**注意:**
- 只有当请求域名(Host)和请求路径(Path)都对应匹配到某条重写规则的 `match_hosts`、`match_paths` 中的一项时,才会将请求域名和请求路径分别重写为该条重写规则的 `rewrite_host`、`rewrite_path`;
- 当配置多条重写规则,将按照规则编写顺序进行匹配;
- 在一条重写规则中,`match_hosts` 和 `match_paths` 按照编写顺序进行匹配;
- `case_sensitive` 也会作用到正则表达式的匹配上。


# 配置示例

## 前缀匹配请求路径

以下配置将请求路径的匹配类型设置为前缀匹配(prefix):

```yaml
rewrite_rules:
- match_path_type: prefix # 前缀匹配
case_sensitive: false
match_hosts:
- foo.bar.com
match_paths:
- /v1/api/get
rewrite_host: prefix.example.com
rewrite_path: /get
```

示例请求 `foo.bar.com/v1/api/get/something` 将被重写为 `prefix.example.com/get`。

## 正则匹配请求路径

以下配置将请求路径的匹配类型设置为正则匹配(regex):

```yaml
rewrite_rules:
- match_path_type: regex # 正则匹配
case_sensitive: false
match_hosts:
- aa.bb.cc
- foo.bar.com
match_paths:
- /abc/(get)
- /(get)/.*\.html
rewrite_host: regex.example.com
rewrite_path: /$1
```

以下示例请求将被重写为 `regex.example.com/get`:
- `foo.bar.com/abc/get`;
- `aa.bb.cc/get/index.html`。


## 通配请求域名

以下配置演示请求域名的精准匹配和最左、最右通配:

```yaml
rewrite_rules:
- match_path_type: exact
case_sensitive: false
match_hosts:
- "hello.world.com" # 精准匹配
- "*.example.com" # 最左通配
- "aa.bb.*" # 最右通配
match_paths:
- /v1/get
- /abc/get/
rewrite_host: wildcard.example.com
rewrite_path: /get
```

以下示例请求将被重写为 `wildcard.example.com/get`:
- `hello.world.com/abc/get/`;
- `my.example.com/v1/get`;
- `aa.bb.com/v1/get`。

## 大小写敏感

以下配置将请求域名和请求路径的匹配设置为大小写敏感:

```yaml
rewrite_rules:
- match_path_type: regex
case_sensitive: true # 大小写敏感
match_hosts:
- xx.yy.ZZ
match_paths:
- /API/(get)
- /test/(get)/.*\.HTML
rewrite_host: case-sensitive.example.com
rewrite_path: /$1
```

以下示例请求将被重写为 `case-sensitive.example.com/get`:
- `xx.yy.ZZ/API/get`;
- `xx.yy.ZZ/test/get/index.HTML`。

而以下示例请求则因为大小写敏感而无法正确匹配:
- `xx.yy.ZZ/api/get`;
- `xx.yy.zz/test/get/index.HTML`。
1 change: 1 addition & 0 deletions plugins/wasm-go/extensions/rewrite/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.0.0
24 changes: 24 additions & 0 deletions plugins/wasm-go/extensions/rewrite/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
version: '3.7'
services:
envoy:
image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/envoy:1.20
depends_on:
- httpbin
networks:
- wasmtest
ports:
- "10000:10000"
- "9901:9901"
volumes:
- ./envoy.yaml:/etc/envoy/envoy.yaml
- ./plugin.wasm:/etc/envoy/main.wasm

httpbin:
image: kennethreitz/httpbin:latest
networks:
- wasmtest
ports:
- "12345:80"

networks:
wasmtest: {}
101 changes: 101 additions & 0 deletions plugins/wasm-go/extensions/rewrite/envoy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
admin:
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
- name: listener_0
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
scheme_header_transformation:
scheme_to_overwrite: https
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/"
name: "foo"
route:
cluster: httpbin
http_filters:
- name: wasmdemo
typed_config:
"@type": type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
name: wasmdemo
vm_config:
runtime: envoy.wasm.runtime.v8
code:
local:
filename: /etc/envoy/main.wasm
configuration:
"@type": "type.googleapis.com/google.protobuf.StringValue"
value: |
{
"rewrite_rules": [
{
"match_path_type": "prefix",
"case_sensitive": false,
"match_hosts": ["foo.bar.com"],
"match_paths": ["/v1/api/get"],
"rewrite_host": "prefix.example.com",
"rewrite_path": "/get"
},
{
"match_path_type": "regex",
"case_sensitive": false,
"match_hosts": ["aa.bb.cc", "foo.bar.com"],
"match_paths": ["/abc/(get)", "/(get)/.*\\.html"],
"rewrite_host": "regex.example.com",
"rewrite_path": "/$1"
},
{
"match_path_type": "exact",
"case_sensitive": false,
"match_hosts": ["hello.world.com", "*.example.com", "aa.bb.*"],
"match_paths": ["/v1/get", "/abc/get/"],
"rewrite_host": "wildcard.example.com",
"rewrite_path": "/get"
},
{
"match_path_type": "regex",
"case_sensitive": true,
"match_hosts": ["xx.yy.ZZ"],
"match_paths": ["/API/(get)", "/test/(get)/.*\\.HTML"],
"rewrite_host": "case-sensitive.example.com",
"rewrite_path": "/$1"
}
]
}
- name: envoy.filters.http.router
clusters:
- name: httpbin
connect_timeout: 30s
type: LOGICAL_DNS
dns_lookup_family: V4_ONLY
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: httpbin
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: httpbin
port_value: 80
16 changes: 16 additions & 0 deletions plugins/wasm-go/extensions/rewrite/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
module github.com/alibaba/higress/plugins/wasm-go/extensions/rewrite

go 1.19

require (
github.com/alibaba/higress/plugins/wasm-go v0.0.0-20230512062358-6b483189acb9
github.com/dlclark/regexp2 v1.10.0
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c
github.com/tidwall/gjson v1.14.3
)

require (
github.com/google/uuid v1.3.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
)
18 changes: 18 additions & 0 deletions plugins/wasm-go/extensions/rewrite/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
github.com/alibaba/higress/plugins/wasm-go v0.0.0-20230512062358-6b483189acb9 h1:j5Q2jO2EB3wPJeyUbpcRMAQb+InH+VeARNpt9lNgH7M=
github.com/alibaba/higress/plugins/wasm-go v0.0.0-20230512062358-6b483189acb9/go.mod h1:AzSnkuon5c26nIePTiJQIAFsKdhkNdncLcTuahpGtQs=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/dlclark/regexp2 v1.10.0 h1:+/GIL799phkJqYW+3YbOd8LCcbHzT0Pbo8zl70MHsq0=
github.com/dlclark/regexp2 v1.10.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c h1:OCUFXVGixHLfNjg6/QYEhv+jHJ5mRGhpEUVFv9eWPJE=
github.com/tetratelabs/proxy-wasm-go-sdk v0.19.1-0.20220822060051-f9d179a57f8c/go.mod h1:5t/pWFNJ9eMyu/K/Z+OeGhDJ9sN9eCo8fc2pyM/Qjg4=
github.com/tidwall/gjson v1.14.3 h1:9jvXn7olKEHU1S9vwoMGliaT8jq1vJ7IH/n9zD9Dnlw=
github.com/tidwall/gjson v1.14.3/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=