Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions docs/docs/Features/sharding.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,13 @@ sidebar_position: 6
```

### 名词解释

SQL 解析: 在分片中,请求到达 Pisa-Proxy 后会首先经过 SQL Parser,将 SQL 解析成 AST。

SQL 改写: 解析结束后,Pisa-Proxy 会根据分片规则对当前 SQL 语句进行改写,生成真实要执行的 SQL 语句。

SQL 路由: Pisa-Proxy 会根据分片规则,将改写好的 SQL 语句路由到后端对应数据源上执行 SQL 语句。

结果归并: SQL 被下推执行之后,Pisa-Proxy 会对查询结果进行归并,并最终返回给客户端。

### SQL 改写介绍
Expand All @@ -37,6 +41,11 @@ SELECT order_id FROM order.t_order WHERE order_id = 1;
```
SELECT order_id FROM order.t_order_00001 WHERE order_id = 1;
```
下图展示了数据查询过程
![](/img/sharding-select.png)

以数据插入为例,数据写入过程如下图:
![](/img/sharding-select.png)

**特别说明: SQL rewrite 在修改标识符计算实际表名时会自动根据分片规则添加表索引,索引规则位 表名_索引,索引位为5位表示。例如:`t_order` 表改写后为 `t_order_00000`。因此用户需要根据实际业务场景先创建好对应的表名**

Expand Down Expand Up @@ -81,9 +90,9 @@ SELECT COUNT(price) AS AVG_DERIVED_COUNT_00000, SUM(price) AS AVG_DERIVED_SUM_00
| 属性 | 值类型 | 是否依赖 | 默认值 | 含义 |
|-----|-------|---------|-------|-----|
| table_name |String|是|无|分片表名|
| actual_datanodes| Vec\<String\>|是|无|后端数据源|
|binding_tables|Vec\<String\>|否|无|暂不支持||
|broadcast_tables|Vec\<String\>|否|无|暂不支持||
| actual_datanodes| array[String]|是|无|后端数据源|
|binding_tables|arrayString]|否|无|暂不支持||
|broadcast_tables|array[String]|否|无|暂不支持||
|table_sharding_algorithm_name|enum|是|无|分片算法|
|table_sharding_column|String|是|无|分片键|
|sharding_count|u64|是|无|分片数|
52 changes: 37 additions & 15 deletions docs/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"prism-react-renderer": "^1.3.1",
"react": "^17.0.2",
"react-dom": "^17.0.2",
"react-github-btn": "^1.2.1"
"react-github-btn": "^1.4.0"
},
"browserslist": {
"production": [
Expand Down
Binary file added docs/static/img/sharding-insert.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/sharding-select.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/versioned_docs/version-v0.3.0/Features/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "特性",
"position": 3
}
51 changes: 51 additions & 0 deletions docs/versioned_docs/version-v0.3.0/Features/loadbalancer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
sidebar_position: 1
---

# 负载均衡

负载均衡模块为 Pisa-Proxy 代理后端节点时,新建链接时对后端节点选取策略的实现。

## 已支持负载均衡策略
- [x] Random
- [x] RoundRobin

## 配置示例

若需要在代理中启用负载均衡,则需要配置 `simple_loadbalance` 块.
```
[proxy.config.simple_loadbalance]
balance_type = "roundrobin"
nodes = ["ds001","ds002"]
```
其中 balance_type: 可选值为 random,roundrobin,默认值为 random;nodes: 选取后端数据源中定义的 `name`.

后端数据源配置:
```
[[mysql.node]]
name = "ds001"
user = "root"
password = "12345678"
db = "test"
host = "127.0.0.1"
port = 3306

[[mysql.node]]
name = "ds002"
user = "root"
password = "12345678"
db = "test"
host = "127.0.0.1"
port = 3307
```

## 模块设计

该模块定义了一个负载均衡的 Trait,封装了在 Pisa-Proxy 中对于负载均衡的构建方式。以及定义了 Random 和 RoundRobin 两种负载均衡策略所要实现的具体方法。

### 代码结构
FILES IN THIS DIRECTORY (loadbalance/src)
balance.rs - 负载均衡 Trait,定义了负载均衡方法和构建负载均衡模块
random_weighted.rs - random weighted 负载均衡策略
roundrobin_weighted.rs - roundrobin weighted 负载均衡策略

51 changes: 51 additions & 0 deletions docs/versioned_docs/version-v0.3.0/Features/mysql-protocol.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
sidebar_position: 2
---
# MySQL 协议

此功能主要为 Pisa-Proxy MySQL 代理的核心组件, 依据 [MySQL 协议](https://dev.mysql.com/doc/internals/en/client-server-protocol.html) 进行实现。该功能主要使用 Rust 生态的 [Tokio](https://github.com/tokio-rs/tokio) 异步运行时框架。其中对网络数据包的读写、协议的编码等操作都通过 Tokio 提供的工具集实现。


## 已支持命令
- [x] COM_INIT_DB
- [x] COM_QUERY
- [x] COM_FIELD_LIST
- [x] COM_QUIT
- [x] COM_PING
- [x] COM_STMT_PREPARE
- [x] COM_STMT_EXECUTE
- [x] COM_STMT_CLOSE
- [x] COM_STMT_RESET

## 支持认证方式
- [x] mysql_native_password
- [ ] sha256_password
- [ ] caching_sha2_password

## 设计说明

本模块共由3部分组成,对应 Pisa-Proxy 中作为客户端和服务端两部分。在 `server` 目录中主要定义了 Pisa-Proxy 作为服务端对客户端请求的处理逻辑。也包含了在 TCP 层的网络数据包的读写操作。在 `client` 目录中定义了 Pisa-Proxy 作为客户端对 MySQL 数据库的建立链接、握手认证和发送客户端命令等操作。

### 代码结构

FILES IN THIS DIRECTORY (protocol/mysql/src/client)
auth.rs - 对 MySQL 登陆认证
codec.rs - 对 MySQL 协议的解码
conn.rs - 对 MySQL 发起链接、握手及命令处理
err.rs - MySQL Error 类型定义
resultset.rs - 一些 MySQL ResultSet 结果处理方法
stmt.rs - 对 MySQL Prepare Statement 处理方法
stream.rs - 对 Tokio TcpSteam 封装

FILES IN THIS DIRECTORY (protocol/mysql/src/server)
conn.rs - 和客户端建链、握手认证处理
packet.rs - 网络数据包的读写操作
stream.rs - 对底层 Tcp 流的封装
tls.rs - MySQL TLS 链接封装

FILES IN THIS DIRECTORY (protocol/mysql/src)
charset.rs - MySQL 字符集声明
err.rs - 协议解析错误处理定义
mysql_const.rs。 - 常量定义
util.rs - 一些通用函数的实现

18 changes: 18 additions & 0 deletions docs/versioned_docs/version-v0.3.0/Features/observability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
sidebar_position: 5
---

# 可观测性

Pisanix 目前在 Pisa-Proxy 处理 SQL 命令的时候采集相关指标,并以 `/metrics` 路径进行暴露。

## 已支持指标
- SQL_PROCESSED_TOTAL: 统计所有执行的 SQL 数量
- SQL_PROCESSED_DURATION: 统计所有 SQL 的执行时间
- SQL_UNDER_PROCESSING: 记录当前正在执行的 SQL 数量

测试效果如下图:

![grafana](/img/grafana.jpg)

下一步将支持更多标签和指标,如 SQL 语句类型、延迟、错误率、TopK、运行时资源等。
Loading