Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

default-database-strategy:hint hint can't use #10699

Closed
stelin opened this issue Jun 7, 2021 · 36 comments
Closed

default-database-strategy:hint hint can't use #10699

stelin opened this issue Jun 7, 2021 · 36 comments
Assignees

Comments

@stelin
Copy link

stelin commented Jun 7, 2021

ShardingSphere-JDBC 4.1.1 default-database-strategy:hint can't use

spring:
shardingsphere:
sharding:
default-data-source-name: demo-1
default-database-strategy:
hint:
algorithm-class-name: com.ai94.mysql.annotation.AnnotationHintShardingAlgorithm
datasource:
names: demo-1,demo-2

datasource one

demo-1:
type: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/db_A?useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC
username: root
password: root

datasource two

demo-2:
type: com.zaxxer.hikari.HikariDataSource
driverClassName: com.mysql.jdbc.Driver
jdbcUrl: jdbc:mysql://localhost:3306/db_B?useUnicode=true&characterEncoding=UTF-8&serverTimezone=UTC
username: root
password: root

@stelin
Copy link
Author

stelin commented Jun 7, 2021

like #5970

@strongduanmu
Copy link
Member

@stelin Can you submit this issue in English?

@stelin stelin changed the title default-database-strategy:hint 不生效 default-database-strategy:hint hint can't use Jun 7, 2021
@stelin
Copy link
Author

stelin commented Jun 7, 2021

@strongduanmu
Only configure default-database-strategy,This method does not seem to be supported!

@strongduanmu
Copy link
Member

strongduanmu commented Jun 10, 2021

Hi @stelin, I used the version 4.1.1 of Sharding-JDBC to test the Hint feature, and there is no problems. The following is my test demo.

public class HintTest {
    
    public static void main(String[] args) throws SQLException {
        DataSource dataSource = new HintTest().getShardingDataSource();
        Connection connection = dataSource.getConnection();
        PreparedStatement preparedStatement = connection.prepareStatement("SELECT * FROM t_order");
        HintManager.getInstance().setDatabaseShardingValue(1);
        ResultSet resultSet = preparedStatement.executeQuery();
        while (resultSet.next()) {
            System.out.println(resultSet.getString("ORDER_ID") + " " + resultSet.getString("USER_ID"));
        }
        HintManager.getInstance().close();
        resultSet.close();
        preparedStatement.close();
        connection.close();
    }
    
    private DataSource getShardingDataSource() throws SQLException {
        ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
        shardingRuleConfig.getTableRuleConfigs().add(getOrderTableRuleConfiguration());
        shardingRuleConfig.getTableRuleConfigs().add(getOrderItemTableRuleConfiguration());
        shardingRuleConfig.getBindingTableGroups().add("t_order, t_order_item");
        shardingRuleConfig.getBroadcastTables().add("t_config");
        shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig(new HintShardingStrategyConfiguration(new AnnotationHintShardingAlgorithm()));
        shardingRuleConfig.setDefaultTableShardingStrategyConfig(new InlineShardingStrategyConfiguration("order_id", "ds_${order_id % 2}"));
        Properties props = new Properties();
        props.setProperty("sql.show", "true");
        return ShardingDataSourceFactory.createDataSource(createDataSourceMap(), shardingRuleConfig, props);
    }
    
    private static KeyGeneratorConfiguration getKeyGeneratorConfiguration() {
        KeyGeneratorConfiguration result = new KeyGeneratorConfiguration("SNOWFLAKE", "order_id");
        return result;
    }
    
    TableRuleConfiguration getOrderTableRuleConfiguration() {
        TableRuleConfiguration result = new TableRuleConfiguration("t_order", "ds_${0..1}.t_order_${0..1}");
        result.setKeyGeneratorConfig(getKeyGeneratorConfiguration());
        return result;
    }
    
    TableRuleConfiguration getOrderItemTableRuleConfiguration() {
        TableRuleConfiguration result = new TableRuleConfiguration("t_order_item", "ds_${0..1}.t_order_item_${0..1}");
        return result;
    }
    
    Map<String, DataSource> createDataSourceMap() {
        Map<String, DataSource> result = new HashMap<>();
        result.put("ds_0", DataSourceUtil.createDataSource("demo_ds_0"));
        result.put("ds_1", DataSourceUtil.createDataSource("demo_ds_1"));
        return result;
    }
}

public final class DataSourceUtil {
    
    private static final String HOST = "localhost";
    
    private static final int PORT = 3306;
    
    private static final String USER_NAME = "root";
    
    private static final String PASSWORD = "123456";
    
    public static DataSource createDataSource(final String dataSourceName) {
        HikariDataSource result = new HikariDataSource();
        result.setDriverClassName(com.mysql.jdbc.Driver.class.getName());
        result.setJdbcUrl(String.format("jdbc:mysql://%s:%s/%s?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8", HOST, PORT, dataSourceName));
        result.setUsername(USER_NAME);
        result.setPassword(PASSWORD);
        return result;
    }
}

public class AnnotationHintShardingAlgorithm implements HintShardingAlgorithm<Integer> {
    
    @Override
    public Collection<String> doSharding(final Collection<String> availableTargetNames, final HintShardingValue<Integer> shardingValue) {
        for (String each : availableTargetNames) {
            if (each.endsWith(String.valueOf(shardingValue.getValues().iterator().next() % 2))) {
                return Collections.singletonList(each);
            }
        }
        return null;
    }
}

@strongduanmu
Copy link
Member

When i set HintManager.getInstance().setDatabaseShardingValue(1);, the route result is:

[main] INFO ShardingSphere-SQL - Logic SQL: SELECT * FROM t_order
[main] INFO ShardingSphere-SQL - SQLStatement: SelectStatementContext(super=CommonSQLStatementContext(sqlStatement=org.apache.shardingsphere.sql.parser.sql.statement.dml.SelectStatement@62b969c4, tablesContext=org.apache.shardingsphere.sql.parser.binder.segment.table.TablesContext@dcc6211), tablesContext=org.apache.shardingsphere.sql.parser.binder.segment.table.TablesContext@dcc6211, projectionsContext=ProjectionsContext(startIndex=7, stopIndex=7, distinctRow=false, projections=[ShorthandProjection(owner=Optional.empty, actualColumns=[ColumnProjection(owner=null, name=order_id, alias=Optional.empty), ColumnProjection(owner=null, name=user_id, alias=Optional.empty), ColumnProjection(owner=null, name=status, alias=Optional.empty), ColumnProjection(owner=null, name=content, alias=Optional.empty), ColumnProjection(owner=null, name=content_new, alias=Optional.empty), ColumnProjection(owner=null, name=content_new_2, alias=Optional.empty), ColumnProjection(owner=null, name=order_name, alias=Optional.empty), ColumnProjection(owner=null, name=item_cd, alias=Optional.empty)])]), groupByContext=org.apache.shardingsphere.sql.parser.binder.segment.select.groupby.GroupByContext@47ec7422, orderByContext=org.apache.shardingsphere.sql.parser.binder.segment.select.orderby.OrderByContext@48535004, paginationContext=org.apache.shardingsphere.sql.parser.binder.segment.select.pagination.PaginationContext@610df783, containsSubquery=false)
[main] INFO ShardingSphere-SQL - Actual SQL: ds_1 ::: SELECT * FROM t_order_0
[main] INFO ShardingSphere-SQL - Actual SQL: ds_1 ::: SELECT * FROM t_order_1

When i set HintManager.getInstance().setDatabaseShardingValue(0);, the route result is:

[main] INFO ShardingSphere-SQL - Logic SQL: SELECT * FROM t_order
[main] INFO ShardingSphere-SQL - SQLStatement: SelectStatementContext(super=CommonSQLStatementContext(sqlStatement=org.apache.shardingsphere.sql.parser.sql.statement.dml.SelectStatement@4441d567, tablesContext=org.apache.shardingsphere.sql.parser.binder.segment.table.TablesContext@3e1624c7), tablesContext=org.apache.shardingsphere.sql.parser.binder.segment.table.TablesContext@3e1624c7, projectionsContext=ProjectionsContext(startIndex=7, stopIndex=7, distinctRow=false, projections=[ShorthandProjection(owner=Optional.empty, actualColumns=[ColumnProjection(owner=null, name=order_id, alias=Optional.empty), ColumnProjection(owner=null, name=user_id, alias=Optional.empty), ColumnProjection(owner=null, name=status, alias=Optional.empty), ColumnProjection(owner=null, name=content, alias=Optional.empty), ColumnProjection(owner=null, name=content_new, alias=Optional.empty), ColumnProjection(owner=null, name=content_new_2, alias=Optional.empty), ColumnProjection(owner=null, name=order_name, alias=Optional.empty), ColumnProjection(owner=null, name=item_cd, alias=Optional.empty)])]), groupByContext=org.apache.shardingsphere.sql.parser.binder.segment.select.groupby.GroupByContext@62b969c4, orderByContext=org.apache.shardingsphere.sql.parser.binder.segment.select.orderby.OrderByContext@dcc6211, paginationContext=org.apache.shardingsphere.sql.parser.binder.segment.select.pagination.PaginationContext@47ec7422, containsSubquery=false)
[main] INFO ShardingSphere-SQL - Actual SQL: ds_0 ::: SELECT * FROM t_order_0
[main] INFO ShardingSphere-SQL - Actual SQL: ds_0 ::: SELECT * FROM t_order_1

When i don't set HintManager.getInstance().setDatabaseShardingValue(0);, the route result is:

[main] INFO ShardingSphere-SQL - Logic SQL: SELECT * FROM t_order
[main] INFO ShardingSphere-SQL - SQLStatement: SelectStatementContext(super=CommonSQLStatementContext(sqlStatement=org.apache.shardingsphere.sql.parser.sql.statement.dml.SelectStatement@6e0cff20, tablesContext=org.apache.shardingsphere.sql.parser.binder.segment.table.TablesContext@359b650b), tablesContext=org.apache.shardingsphere.sql.parser.binder.segment.table.TablesContext@359b650b, projectionsContext=ProjectionsContext(startIndex=7, stopIndex=7, distinctRow=false, projections=[ShorthandProjection(owner=Optional.empty, actualColumns=[ColumnProjection(owner=null, name=order_id, alias=Optional.empty), ColumnProjection(owner=null, name=user_id, alias=Optional.empty), ColumnProjection(owner=null, name=status, alias=Optional.empty), ColumnProjection(owner=null, name=content, alias=Optional.empty), ColumnProjection(owner=null, name=content_new, alias=Optional.empty), ColumnProjection(owner=null, name=content_new_2, alias=Optional.empty), ColumnProjection(owner=null, name=order_name, alias=Optional.empty), ColumnProjection(owner=null, name=item_cd, alias=Optional.empty)])]), groupByContext=org.apache.shardingsphere.sql.parser.binder.segment.select.groupby.GroupByContext@6a9b0a6f, orderByContext=org.apache.shardingsphere.sql.parser.binder.segment.select.orderby.OrderByContext@10667848, paginationContext=org.apache.shardingsphere.sql.parser.binder.segment.select.pagination.PaginationContext@191a709b, containsSubquery=false)
[main] INFO ShardingSphere-SQL - Actual SQL: ds_0 ::: SELECT * FROM t_order_0
[main] INFO ShardingSphere-SQL - Actual SQL: ds_0 ::: SELECT * FROM t_order_1
[main] INFO ShardingSphere-SQL - Actual SQL: ds_1 ::: SELECT * FROM t_order_0
[main] INFO ShardingSphere-SQL - Actual SQL: ds_1 ::: SELECT * FROM t_order_1

So I want to know what is your configuration? What is the abnormal phenomenon?

@stelin
Copy link
Author

stelin commented Jun 10, 2021

I work will like your method. but I do not configure TableRuleConfiguration, becase only want to sharding for datasource.If must to configure table rules,It's very troublesome for many tables(200+)

@stelin
Copy link
Author

stelin commented Jun 10, 2021

Only one rule for database.Has not table rules.
image

@stelin
Copy link
Author

stelin commented Jun 10, 2021

If must to configure table rules like below.
image

@stelin
Copy link
Author

stelin commented Jun 10, 2021

It's very troublesome, because every table must to be configured.

@stelin
Copy link
Author

stelin commented Jun 10, 2021

We only want to switch datasource(database), It has nothing to do with tables.

@strongduanmu
Copy link
Member

@stelin Do you mean that your tables are all non-sharding tables, so TableRuleConfiguration is not configured? It should be noted that DefaultDatabaseShardingStrategyConfig is for the configuration of sharding tables, and it is inappropriate to use in this scenario.

@strongduanmu
Copy link
Member

@stelin In version 4.x, you need to configure shardingRuleConfig.setDefaultDataSourceName("ds_0"); to meet your needs. In version 5.x, we introduced the concept of single table, you can route these tables automatically without any configuration.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

@strongduanmu

Yes my tables are all non-sharding tables. Now in version 4.x is not supported! But version 5.x do not released. If 5.x can use?

@strongduanmu
Copy link
Member

@stelin In version 4.x, you can configure shardingRuleConfig.setDefaultDataSourceName("ds_0");

@stelin
Copy link
Author

stelin commented Jun 11, 2021

I configured shardingRuleConfig.setDefaultDataSourceName("ds_0"), But can not to switch datasource;

@stelin
Copy link
Author

stelin commented Jun 11, 2021

Want to switch datasource by application context(like http header)

@stelin
Copy link
Author

stelin commented Jun 11, 2021

hint:algorithm-class-name can not worker for this.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

Single table can not solved.My scenario is one datasource has many tables. Olny sharding for database(datasource). Tables not to sharding

@strongduanmu
Copy link
Member

Single table can not solved.My scenario is one datasource has many tables. Olny sharding for database(datasource). Tables not to sharding

@stelin If you need sharding, then you need to configure sharding rules. You can either shard the database and the table at the same time, or you can only shard the database or only the table. Without configuration rules, the Hint sharding algorithm will definitely not take effect.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

If only want to sharding for database(datasource), then with configuration table rules is very troublesome. If add one table ,must to add configuration

@strongduanmu
Copy link
Member

@stelin If you have a better solution about sharding api, please submit an issue to the community for discussion.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

Now can not use shardingjdbc to my scenario. You have solution?

@strongduanmu
Copy link
Member

@stelin As I said before, configure sharding rules.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

Understanded

@strongduanmu
Copy link
Member

@stelin If this issue has been resolved, please close it. If you have a better idea, please submit an issue to discuss about sharding api.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

A a better method to supported switch datasource without any table sharding rules configuration.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

But now is not supported.

@strongduanmu
Copy link
Member

@stelin Your idea is to change the sharding api and the underlying implementation, which needs to be discussed by the community.
The previous demo has given an example where the hint can be executed normally after configuring the sharding rules. So I think this issue can be closed. If you have configured sharding rules and encounter other problems, please submit a new issue and upload the corresponding demo.

@stelin
Copy link
Author

stelin commented Jun 11, 2021

Thanks

@stelin
Copy link
Author

stelin commented Jun 17, 2021

@strongduanmu

What are the results of your discussion?

Can supported table match rules, like below order_*

shardingRule:
  tables:
   order_*:
        actualDataNodes: demo_ds_${0..1}.t_order_${0..1}
        databaseStrategy:
          hint:
            algorithmClassName: io.shardingsphere.userAlgo.HintAlgorithm
        tableStrategy:
          hint:
            algorithmClassName: io.shardingsphere.userAlgo.HintAlgorithm

@wanlongz
Copy link

@stelin How did you solve it in the end? I have the same problem

@tuziaifendou
Copy link

@stelin In version 4.x, you need to configure shardingRuleConfig.setDefaultDataSourceName("ds_0"); to meet your needs. In version 5.x, we introduced the concept of single table, you can route these tables automatically without any configuration.

In version 4.1.1,when i add spring.shardingsphere.sharding.default-data-source-name=ds0.when start service,meta data load cost about 204022 milliseconds,then service starts very slowly. how to solution?

@tuziaifendou
Copy link

@stelin How did you solve it in the end? I have the same problem

I have the same problem too

@strongduanmu
Copy link
Member

@stelin How did you solve it in the end? I have the same problem

I have the same problem too

@tuziaifendou I think you can submit a new issue to describe your problem.

@tuziaifendou
Copy link

tuziaifendou commented Jan 7, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants