Skip to content

Commit

Permalink
Keep data consistent in same transaction for readwrite-splitting only (
Browse files Browse the repository at this point in the history
  • Loading branch information
terrymanu committed Nov 1, 2021
1 parent e775f20 commit 6b3095a
Show file tree
Hide file tree
Showing 71 changed files with 63 additions and 246 deletions.
Expand Up @@ -6,18 +6,12 @@ weight = 2
## 支持项

* 提供一主多从的读写分离配置,可独立使用,也可配合数据分片使用;
* 独立使用读写分离支持 SQL 透传
* 事务中的数据读写均用主库
* 基于 Hint 的强制主库路由。

## 不支持项

* 主库和从库的数据同步;
* 主库和从库的数据同步延迟导致的数据不一致;
* 主库双写或多写;
* 跨主库和从库之间的事务的数据不一致。主从模型中,事务中读写均用主库。

## 可选项

| 配置项 | 详细说明 | 默认值 | 适用范围 |
|:--------------- |:-------------------------------------------------------------------------------------- |:------ |:------------------ |
| queryConsistent | 配置为 `true` 时,若在同一线程且同一数据库连接内有写入操作,以后的读操作均从主库读取,用于保证数据一致性 | false | ShardingSphere-JDBC |
* 主库多写;
* 主从库间的事务一致性。主从模型中,事务中的数据读写均用主库。
Expand Up @@ -6,7 +6,7 @@ weight = 2
## Supported

* Provide the readwrite-splitting configuration of one primary database with multiple replica databases, which can be used alone or with sharding table and database;
* Support SQL pass-through in independent use of readwrite-splitting;
* Primary nodes need to be used for both reading and writing in the transaction;
* Forcible primary database route based on SQL Hint;

## Unsupported
Expand All @@ -15,10 +15,4 @@ weight = 2
* Data inconsistency caused by replication delay between databases;
* Double or multiple primary databases to provide write operation;
* The data for transaction across primary and replica nodes are inconsistent;
In the readwrite-splitting model, the primary nodes need to be used for both reading and writing in the transaction.

## Optional

| Item | Description | Default | Range Of Application |
|:--------------- |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |:------- |:-------------------- |
| queryConsistent | When the value is `true`, if there is write operation in the same thread and database connection, all the following read operations are from the write database to ensure data consistency | false | ShardingSphere-JDBC |
In the readwrite-splitting model, primary nodes need to be used for both reading and writing in the transaction.
Expand Up @@ -142,7 +142,6 @@ Namespace: [http://shardingsphere.apache.org/schema/shardingsphere/readwrite-spl
| write-data-source-name | Attribute | Write data source name |
| read-data-source-names | Attribute | Read data source names, multiple data source names separated with comma |
| load-balance-algorithm-ref | Attribute | Load balance algorithm name |
| query-consistent | Attribute | Whether to enable query consistent routing |

\<readwrite-splitting:load-balance-algorithm />

Expand Down
Expand Up @@ -26,7 +26,6 @@ weight = 2
| writeDataSourceName | String | 写库数据源名称 | - |
| readDataSourceNames (+) | Collection\<String\> | 读库数据源名称列表 | - |
| loadBalancerName (?) | String | 读库负载均衡算法名称 | 轮询负载均衡算法 |
| queryConsistent (?) | boolean | 是否启用查询一致性路由 | false |

算法类型的详情,请参见[内置负载均衡算法列表](/cn/user-manual/shardingsphere-jdbc/configuration/built-in-algorithm/load-balance)
查询一致性路由的详情,请参见[使用规范](/cn/features/readwrite-splitting/use-norms)
Expand Up @@ -26,7 +26,6 @@ Attributes:
| writeDataSourceName | String | Write sources source name | - |
| readDataSourceNames (+) | Collection\<String\> | Read sources source name list | - |
| loadBalancerName (?) | String | Load balance algorithm name of replica sources | Round robin load balance algorithm |
| queryConsistent (?) | boolean | Whether to enable query consistent routing | false |

Please refer to [Built-in Load Balance Algorithm List](/en/user-manual/shardingsphere-jdbc/configuration/built-in-algorithm/load-balance) for more details about type of algorithm.
Please refer to [Use Norms](/en/features/readwrite-splitting/use-norms) for more details about query consistent routing.
Expand Up @@ -11,7 +11,6 @@ spring.shardingsphere.datasource.names= # 省略数据源配置,请参考使
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.write-data-source-name= # 写数据源名称
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.read-data-source-names= # 读数据源名称,多个从数据源用逗号分隔
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.load-balancer-name= # 负载均衡算法名称
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.query-consistent= # 是否启用查询一致性路由

# 负载均衡算法配置
spring.shardingsphere.rules.readwrite-splitting.load-balancers.<load-balance-algorithm-name>.type= # 负载均衡算法类型
Expand Down
Expand Up @@ -11,7 +11,6 @@ spring.shardingsphere.datasource.names= # Omit the data source configuration, pl
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.primary-data-source-name= # Write data source name
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.replica-data-source-names= # Read data source names, multiple data source names separated with comma
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.load-balancer-name= # Load balance algorithm name
spring.shardingsphere.rules.readwrite-splitting.data-sources.<readwrite-splitting-data-source-name>.query-consistent= # Whether to enable query consistent routing

# Load balance algorithm configuration
spring.shardingsphere.rules.readwrite-splitting.load-balancers.<load-balance-algorithm-name>.type= # Load balance algorithm type
Expand Down
Expand Up @@ -22,13 +22,12 @@ weight = 2
| write-data-source-name | 属性 | 写数据源名称 |
| read-data-source-names | 属性 | 读数据源名称,多个读数据源用逗号分隔 |
| load-balance-algorithm-ref | 属性 | 负载均衡算法名称 |
| query-consistent | 属性 | 是否启用查询一致性路由 |


\<readwrite-splitting:load-balance-algorithm />

| *名称* | *类型* | *说明* |
| --------- | ----- | ----------------- |
| *名称* | *类型* | *说明* |
| --------- | ----- | ---------------- |
| id | 属性 | 负载均衡算法名称 |
| type | 属性 | 负载均衡算法类型 |
| props (?) | 标签 | 负载均衡算法属性配置 |
Expand Down
Expand Up @@ -22,7 +22,6 @@ Namespace: [http://shardingsphere.apache.org/schema/shardingsphere/readwrite-spl
| write-data-source-name | Attribute | Write data source name |
| read-data-source-names | Attribute | Read data source names, multiple data source names separated with comma |
| load-balance-algorithm-ref | Attribute | Load balance algorithm name |
| query-consistent | Attribute | Whether to enable query consistent routing |

\<readwrite-splitting:load-balance-algorithm />

Expand Down
Expand Up @@ -16,7 +16,6 @@ rules:
readDataSourceNames:
- <read-data_source-name> (+) # 读库数据源名称
loadBalancerName: # 负载均衡算法名称
queryConsistent: # 是否启用查询一致性路由

# 负载均衡算法配置
loadBalancers:
Expand Down
Expand Up @@ -16,7 +16,6 @@ rules:
readDataSourceNames:
- <read-data-source-name> (+) # Read data source name
loadBalancerName: # Load balance algorithm name
queryConsistent: # Whether to enable query consistent routing

# Load balance algorithm configuration
loadBalancers:
Expand Down
Expand Up @@ -83,9 +83,9 @@ private static ShardingTableRuleConfiguration createOrderItemTableRuleConfigurat

private static ReadwriteSplittingRuleConfiguration createReadwriteSplittingConfiguration() {
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration1 = new ReadwriteSplittingDataSourceRuleConfiguration(
"ds_0", "", "demo_write_ds_0", Arrays.asList("demo_write_ds_0_read_0", "demo_write_ds_0_read_1"), null, false);
"ds_0", "", "demo_write_ds_0", Arrays.asList("demo_write_ds_0_read_0", "demo_write_ds_0_read_1"), null);
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration2 = new ReadwriteSplittingDataSourceRuleConfiguration(
"ds_1", "", "demo_write_ds_1", Arrays.asList("demo_write_ds_1_read_0", "demo_write_ds_1_read_1"), null, false);
"ds_1", "", "demo_write_ds_1", Arrays.asList("demo_write_ds_1_read_0", "demo_write_ds_1_read_1"), null);
return new ReadwriteSplittingRuleConfiguration(Arrays.asList(dataSourceConfiguration1, dataSourceConfiguration2), Collections.emptyMap());
}

Expand Down
Expand Up @@ -83,9 +83,9 @@ private static ShardingTableRuleConfiguration createOrderItemTableRuleConfigurat

private static ReadwriteSplittingRuleConfiguration createReadwriteSplittingConfiguration() {
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration1 = new ReadwriteSplittingDataSourceRuleConfiguration(
"ds_0", "", "demo_write_ds_0", Arrays.asList("demo_write_ds_0_read_0", "demo_write_ds_0_read_1"), null, false);
"ds_0", "", "demo_write_ds_0", Arrays.asList("demo_write_ds_0_read_0", "demo_write_ds_0_read_1"), null);
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration2 = new ReadwriteSplittingDataSourceRuleConfiguration(
"ds_1", "", "demo_write_ds_1", Arrays.asList("demo_write_ds_1_read_0", "demo_write_ds_1_read_1"), null, false);
"ds_1", "", "demo_write_ds_1", Arrays.asList("demo_write_ds_1_read_0", "demo_write_ds_1_read_1"), null);
return new ReadwriteSplittingRuleConfiguration(Arrays.asList(dataSourceConfiguration1, dataSourceConfiguration2), Collections.emptyMap());
}

Expand Down
Expand Up @@ -114,12 +114,10 @@ rules:
writeDataSourceName: write_ds_0
readDataSourceNames: [write_ds_0_read_0, write_ds_0_read_1]
loadBalancerName: roundRobin
queryConsistent: true
ds_1:
writeDataSourceName: write_ds_1
readDataSourceNames: [write_ds_1_read_0, write_ds_1_read_1]
loadBalancerName: roundRobin
queryConsistent: false
loadBalancers:
roundRobin:
type: ROUND_ROBIN
Expand Down
Expand Up @@ -90,9 +90,7 @@ spring.shardingsphere.rules.sharding.sharding-algorithms.t-order-item-inline.pro

spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-0.write-data-source-name=write-ds-0
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-0.read-data-source-names=write-ds-0-read-0,write-ds-0-read-1
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-0.query-consistent=true
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-1.write-data-source-name=write-ds-1
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-1.read-data-source-names=write-ds-1-read-0,write-ds-1-read-1
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-1.query-consistent=false

spring.shardingsphere.props.sql-show=true
Expand Up @@ -91,7 +91,5 @@ spring.shardingsphere.rules.sharding.sharding-algorithms.t-order-item-inline.pro

spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-0.write-data-source-name=write-ds-0
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-0.read-data-source-names=write-ds-0-read-0, write-ds-0-read-1
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-0.query-consistent=true
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-1.write-data-source-name=write-ds-1
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-1.read-data-source-names=write-ds-1-read-0, write-ds-1-read-1
spring.shardingsphere.rules.readwrite-splitting.data-sources.ds-1.query-consistent=false
Expand Up @@ -150,8 +150,8 @@
</sharding:broadcast-table-rules>
</sharding:rule>
<readwrite-splitting:rule id="readWriteSplittingRule">
<readwrite-splitting:data-source-rule id="demo_pr_ds_0" write-data-source-name="demo_write_ds_0" read-data-source-names="demo_write_ds_0_read_0, demo_write_ds_0_read_1" load-balance-algorithm-ref="randomStrategy" query-consistent="false" />
<readwrite-splitting:data-source-rule id="demo_pr_ds_1" write-data-source-name="demo_write_ds_1" read-data-source-names="demo_write_ds_1_read_0, demo_write_ds_1_read_1" load-balance-algorithm-ref="randomStrategy" query-consistent="true" />
<readwrite-splitting:data-source-rule id="demo_pr_ds_0" write-data-source-name="demo_write_ds_0" read-data-source-names="demo_write_ds_0_read_0, demo_write_ds_0_read_1" load-balance-algorithm-ref="randomStrategy" />
<readwrite-splitting:data-source-rule id="demo_pr_ds_1" write-data-source-name="demo_write_ds_1" read-data-source-names="demo_write_ds_1_read_0, demo_write_ds_1_read_1" load-balance-algorithm-ref="randomStrategy" />
</readwrite-splitting:rule>

<shardingsphere:data-source id="shardingDataSource" data-source-names="demo_write_ds_0,demo_write_ds_1,demo_write_ds_0_read_0, demo_write_ds_0_read_1,demo_write_ds_1_read_0, demo_write_ds_1_read_1" rule-refs="shardingRule, readWriteSplittingRule">
Expand Down
Expand Up @@ -43,7 +43,7 @@ public LocalReadwriteSplittingConfiguration(final ModeConfiguration modeConfig)
@Override
public DataSource getDataSource() throws SQLException {
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfig = new ReadwriteSplittingDataSourceRuleConfiguration(
"demo_readwrite_splitting_ds", "", "demo_write_ds", Arrays.asList("demo_read_ds_0", "demo_read_ds_1"), null, false);
"demo_readwrite_splitting_ds", "", "demo_write_ds", Arrays.asList("demo_read_ds_0", "demo_read_ds_1"), null);
ReadwriteSplittingRuleConfiguration ruleConfig = new ReadwriteSplittingRuleConfiguration(Collections.singleton(dataSourceConfig), Collections.emptyMap());
return ShardingSphereDataSourceFactory.createDataSource(modeConfig, createDataSourceMap(), Collections.singleton(ruleConfig), new Properties());
}
Expand Down
Expand Up @@ -86,9 +86,9 @@ private static KeyGenerateStrategyConfiguration getKeyGeneratorConfiguration() {

private ReadwriteSplittingRuleConfiguration getReadwriteSplittingRuleConfiguration() {
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration1 = new ReadwriteSplittingDataSourceRuleConfiguration(
"ds_0", "", "demo_write_ds_0", Arrays.asList("demo_write_ds_0_read_0", "demo_write_ds_0_read_1"), null, false);
"ds_0", "", "demo_write_ds_0", Arrays.asList("demo_write_ds_0_read_0", "demo_write_ds_0_read_1"), null);
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfiguration2 = new ReadwriteSplittingDataSourceRuleConfiguration(
"ds_1", "", "demo_write_ds_1", Arrays.asList("demo_write_ds_1_read_0", "demo_write_ds_1_read_1"), null, false);
"ds_1", "", "demo_write_ds_1", Arrays.asList("demo_write_ds_1_read_0", "demo_write_ds_1_read_1"), null);
return new ReadwriteSplittingRuleConfiguration(Arrays.asList(dataSourceConfiguration1, dataSourceConfiguration2), Collections.emptyMap());
}

Expand Down
Expand Up @@ -36,7 +36,7 @@ public final class ReadwriteSplittingConfiguration implements ExampleConfigurati
@Override
public DataSource getDataSource() throws SQLException {
ReadwriteSplittingDataSourceRuleConfiguration dataSourceConfig = new ReadwriteSplittingDataSourceRuleConfiguration(
"demo_read_query_ds", "", "demo_write_ds", Arrays.asList("demo_read_ds_0", "demo_read_ds_1"), null, false);
"demo_read_query_ds", "", "demo_write_ds", Arrays.asList("demo_read_ds_0", "demo_read_ds_1"), null);
ReadwriteSplittingRuleConfiguration ruleConfig = new ReadwriteSplittingRuleConfiguration(Collections.singleton(dataSourceConfig), Collections.emptyMap());
return ShardingSphereDataSourceFactory.createDataSource(createDataSourceMap(), Collections.singleton(ruleConfig), new Properties());
}
Expand Down
Expand Up @@ -59,7 +59,7 @@
<readwrite-splitting:load-balance-algorithm id="randomStrategy" type="RANDOM" />

<readwrite-splitting:rule id="readWriteSplittingRule">
<readwrite-splitting:data-source-rule id="demo_ds" write-data-source-name="demo_write_ds" read-data-source-names="demo_read_ds_0, demo_read_ds_1" load-balance-algorithm-ref="randomStrategy" query-consistent="true" />
<readwrite-splitting:data-source-rule id="demo_ds" write-data-source-name="demo_write_ds" read-data-source-names="demo_read_ds_0, demo_read_ds_1" load-balance-algorithm-ref="randomStrategy" />
</readwrite-splitting:rule>

<shardingsphere:data-source id="readWriteSplittingDataSource" data-source-names="demo_write_ds, demo_read_ds_0, demo_read_ds_1" rule-refs="readWriteSplittingRule" />
Expand Down
Expand Up @@ -55,7 +55,6 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false

loadBalancers:
read_balance:
Expand Down
Expand Up @@ -55,7 +55,6 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false

loadBalancers:
read_balance:
Expand Down
Expand Up @@ -55,7 +55,6 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false

loadBalancers:
read_balance:
Expand Down
Expand Up @@ -55,7 +55,6 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false

loadBalancers:
read_balance:
Expand Down
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN
Expand Up @@ -55,8 +55,7 @@ rules:
- read_ds_0
- read_ds_1
loadBalancerName: read_balance
queryConsistent: false


loadBalancers:
read_balance:
type: ROUND_ROBIN

0 comments on commit 6b3095a

Please sign in to comment.