Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I use multiple sharding rules for the same table? #31374

Closed
LostMyth opened this issue May 24, 2024 · 8 comments
Closed

How can I use multiple sharding rules for the same table? #31374

LostMyth opened this issue May 24, 2024 · 8 comments

Comments

@LostMyth
Copy link

Shardingsphere-JDBC 5.5.0 does not support multiple sharding rules for the same table. In my business scenarios, most of the scenarios use standard rule, but in some special scenarios, I need to use hint rule. How can I configure it so that I can use two kinds of sharding rules? I read the introduction of forced routing in the official website, if I configure hint rule, I can't use other sharding rules, right?

i tried this

    tables:
      t_order:
        actualDataNodes: ds_$->{0..2}.t_order_$->{0..3}
        databaseStrategy:
          standard:
            shardingColumn: member_id
            shardingAlgorithmName: datasourceClassBased
          hint:
            shardingAlgorithmName: datasourceHintClassBased
        tableStrategy:
          standard:
            shardingColumn: member_id
            shardingAlgorithmName: tableClassBased
          hint:
            shardingAlgorithmName: tableHintClassBased

it got error: Only allowed 0 or 1 sharding strategy configuration.

and i tried this

    tables:
      t_order:
        actualDataNodes: ds_$->{0..2}.t_order_$->{0..3}
        databaseStrategy:
          standard:
            shardingColumn: member_id
            shardingAlgorithmName: datasourceClassBased
        tableStrategy:
          standard:
            shardingColumn: member_id
            shardingAlgorithmName: tableClassBased
      t_order_hint:
        actualDataNodes: ds_$->{0..2}.t_order_$->{0..3}
        databaseStrategy:
          hint:
            shardingAlgorithmName: datasourceHintClassBased
        tableStrategy:
          hint:
            shardingAlgorithmName: tableHintClassBased

it got error: Same actual data node cannot be configured in multiple logic tables in same database, logical table 't_order', actual data node 'ds_0.t_order_0'.

@terrymanu
Copy link
Member

The data in a database can only have one storage structure. It maybe not a good practice to having more than one sharding strategies in the same table.

@LostMyth
Copy link
Author

The data in a database can only have one storage structure. It maybe not a good practice to having more than one sharding strategies in the same table.
Thanks very much for answer my question.
I am using shardingjdbc-4.1.1 in my project, in this version it is possible to configure two logical tables for the same physical table, corresponding to standard routing, and forced routing respectively. The standard route is used in normal business, and the forced route is used when exporting table data. Now I want to upgrade to Shardingsphere-jdbc-5.5.0, if I can't use forced route to scan table data and then export, I have to do standard route to scan each table. This will result in wasted queries, so is there any other method other than forced routing that will help to export all the sub-table data?

@SoLoHiC
Copy link

SoLoHiC commented Jun 6, 2024

We have a similar scenario too. For most of the requests(, which have a sharding key), it's supposed to be covered by the inline strategy. But for certain requests (, from scheduled jobs and it's about to scan the whole table), we'd want it to be split into multi-threads and each thread could use a hint to specify a certain sharding table to scan. Now the problem is, once we configured an inline strategy for the table, a hint strategy couldn't be used programmatically.

@LostMyth
Copy link
Author

LostMyth commented Jun 6, 2024

We have a similar scenario too. For most of the requests(, which have a sharding key), it's supposed to be covered by the inline strategy. But for certain requests (, from scheduled jobs and it's about to scan the whole table), we'd want it to be split into multi-threads and each thread could use a hint to specify a certain sharding table to scan. Now the problem is, once we configured an inline strategy for the table, a hint strategy couldn't be used programmatically.

yes, lower version can do that , but in 5.5.0 can't do that . if use 5.5.0, maybe can only use inline strategy to scan the whole table.

@Janche
Copy link

Janche commented Jun 13, 2024

我们也遇到同样的问题了,多余99%的情况采用 inline方式分表,极少数需要通过 hint的方式指定查询原逻辑表(没有分表后缀),结果发现 hint 方式不生效,版本5.2.1

@Janche
Copy link

Janche commented Jun 14, 2024

我们也有类似的情况。对于大多数请求(具有分片键的请求),它应该由内联策略覆盖。但对于某些请求(来自计划作业并且即将扫描整个表),我们希望将其拆分为多线程,并且每个线程可以使用提示来指定要扫描的特定分片表。现在的问题是,一旦我们为表配置了内联策略,就无法以编程方式使用提示策略。

是的,低版本可以,但是5.5.0不可以。如果使用5.5.0,可能只能使用内联策略来扫描全表。

请问哪一个版本可以?

@LostMyth
Copy link
Author

我们也有类似的情况。对于大多数请求(具有分片键的请求),它应该由内联策略覆盖。但对于某些请求(来自计划作业并且即将扫描整个表),我们希望将其拆分为多线程,并且每个线程可以使用提示来指定要扫描的特定分片表。现在的问题是,一旦我们为表配置了内联策略,就无法以编程方式使用提示策略。

是的,低版本可以,但是5.5.0不可以。如果使用5.5.0,可能只能使用内联策略来扫描全表。

请问哪一个版本可以?

sharding-jdbc 4.1.1 可以,现在用的是shardingsphere-jdbc版本了,我想从sharding-jdbc 4.1.1 升级到shardingsphere-jdbc 5.5.0, 目前遇到上述问题。

@terrymanu
Copy link
Member

The data in a database can only have one storage structure. It maybe not a good practice to having more than one sharding strategies in the same table.

Just as my answer, It's best not to add unnecessary complexity to ShardingSphere for some non-standard requirements. We all hope that ShardingSphere remains concise and clear.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants