Shadow sample throws exception 'Table does not exist'

[bobbyz007]

Question

When I start a sample with shadow feature, it complains 'Table not exist', are there some configs that i'm missing or making wrong?

version: 5.5.0

shadow config: config.yaml

dataSources:
  ds:
    driverClassName: com.mysql.cj.jdbc.Driver
    dataSourceClassName: com.alibaba.druid.pool.DruidDataSource
    # 使用druid必须用url，而不是jdbcUrl
    url: jdbc:mysql://127.0.0.1:3306/ds?characterEncoding=utf-8
    username: mysql
    password: mysql
  ds_shadow:
    driverClassName: com.mysql.cj.jdbc.Driver
    dataSourceClassName: com.alibaba.druid.pool.DruidDataSource
    # 使用druid必须用url，而不是jdbcUrl
    url: jdbc:mysql://127.0.0.1:3306/ds_shadow?characterEncoding=utf-8
    username: mysql
    password: mysql

rules:
  - !SHADOW
    dataSources:
      shadowDataSource:
        productionDataSourceName: ds
        shadowDataSourceName: ds_shadow
    tables:
      t_user:
        dataSourceNames:
          - shadowDataSource
        shadowAlgorithmNames:
          - user_id_insert_match_algorithm
          - user_id_delete_match_algorithm
          - user_id_select_match_algorithm
          - sql_hint_algorithm
    shadowAlgorithms:
      user_id_insert_match_algorithm:
        type: REGEX_MATCH
        props:
          operation: insert
          column: user_type
          regex: "[1]"
      user_id_delete_match_algorithm:
        type: REGEX_MATCH
        props:
          operation: delete
          column: user_type
          regex: "[1]"
      user_id_select_match_algorithm:
        type: REGEX_MATCH
        props:
          operation: select
          column: user_type
          regex: "[1]"
      sql_hint_algorithm:
        type: SQL_HINT
#打印sql
props:
  sql-show: true

throws exception:

org.apache.shardingsphere.infra.exception.kernel.metadata.TableNotFoundException: Table or view 't_user' does not exist.
	at org.apache.shardingsphere.infra.binder.segment.from.impl.SimpleTableSegmentBinder.lambda$checkTableExists$4(SimpleTableSegmentBinder.java:151) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.exception.core.ShardingSpherePreconditions.checkState(ShardingSpherePreconditions.java:41) ~[shardingsphere-infra-exception-core-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.segment.from.impl.SimpleTableSegmentBinder.checkTableExists(SimpleTableSegmentBinder.java:148) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.segment.from.impl.SimpleTableSegmentBinder.bind(SimpleTableSegmentBinder.java:84) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.statement.dml.InsertStatementBinder.lambda$bind$0(InsertStatementBinder.java:58) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at java.base/java.util.Optional.ifPresent(Optional.java:178) ~[?:?]
	at org.apache.shardingsphere.infra.binder.statement.dml.InsertStatementBinder.bind(InsertStatementBinder.java:58) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.statement.dml.InsertStatementBinder.bind(InsertStatementBinder.java:48) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.engine.SQLBindEngine.bindDMLStatement(SQLBindEngine.java:85) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.engine.SQLBindEngine.bind(SQLBindEngine.java:72) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.infra.binder.engine.SQLBindEngine.bind(SQLBindEngine.java:63) ~[shardingsphere-infra-binder-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.<init>(ShardingSpherePreparedStatement.java:207) ~[shardingsphere-jdbc-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.driver.jdbc.core.statement.ShardingSpherePreparedStatement.<init>(ShardingSpherePreparedStatement.java:172) ~[shardingsphere-jdbc-5.5.0.jar:5.5.0]
	at org.apache.shardingsphere.driver.jdbc.core.connection.ShardingSphereConnection.prepareStatement(ShardingSphereConnection.java:94) ~[shardingsphere-jdbc-5.5.0.jar:5.5.0]
	at com.zaxxer.hikari.pool.ProxyConnection.prepareStatement(ProxyConnection.java:327) ~[HikariCP-5.0.1.jar:?]
	at com.zaxxer.hikari.pool.HikariProxyConnection.prepareStatement(HikariProxyConnection.java) ~[HikariCP-5.0.1.jar:?]
	at org.apache.ibatis.executor.statement.PreparedStatementHandler.instantiateStatement(PreparedStatementHandler.java:88) ~[mybatis-3.5.14.jar:3.5.14]

[TherChenYang]

@bobbyz007 Thank you for your feedback, I have verified the same issue in master, and will investigate this issue and provide a response.

[linghengqian]

• I'm just providing some additional information.
• In fact, NativeTest under GraalVM Native Image reuses the example file, and nativeTest is normal. Reference https://github.com/apache/shardingsphere/blob/b3fed4cb1834ca6089c13b72db162cd0c2124f6f/test/native/src/test/resources/test-native/yaml/features/shadow.yaml .
• The relevant unit tests are located at https://github.com/apache/shardingsphere/blob/830dfc8d8e63909a59ef8b1cd45ba7bf8b1a6ef4/test/native/src/test/java/org/apache/shardingsphere/test/natived/jdbc/features/ShadowTest.java . You can execute this unit test manually in IntelliJ IDEA. I would say this looks like the example is not initializing part of the table accurately.

[TherChenYang]

• When the initEnvironment() method in ShawTest is commented out, directly executing selectAll will still result in the same error. When using the shadow module, we do not restrict that table initialization must be done within ShardingSphere.
• I think we should load the tables that have already been configured in the configuration file. See PR Fix shadow moudle throw 'Table does not exist' exception #31378.

[zhaojinchao95]

@bobbyz007 Hello，maybe you should add single configuration. such as:

databaseName: shadow_db

dataSources:
  ds:
    url: jdbc:mysql://127.0.0.1:3306/ds?useSSL=false
    username: root
    password: 123456
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50
    minPoolSize: 1
  ds_shadow:
    url: jdbc:mysql://127.0.0.1:3306/ds_shadow?useSSL=false
    username: root
    password: 123456
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50
    minPoolSize: 1

rules:
  - !SINGLE
    tables:
      - "*.*"
  - !SHADOW
    dataSources:
      shadowDataSource:
        productionDataSourceName: ds
        shadowDataSourceName: ds_shadow
    tables:
      t_user:
        dataSourceNames:
          - shadowDataSource
        shadowAlgorithmNames:
          - user_id_insert_match_algorithm
          - user_id_delete_match_algorithm
          - user_id_select_match_algorithm
          - sql_hint_algorithm
    shadowAlgorithms:
      user_id_insert_match_algorithm:
        type: REGEX_MATCH
        props:
          operation: insert
          column: user_type
          regex: "[1]"
      user_id_delete_match_algorithm:
        type: REGEX_MATCH
        props:
          operation: delete
          column: user_type
          regex: "[1]"
      user_id_select_match_algorithm:
        type: REGEX_MATCH
        props:
          operation: select
          column: user_type
          regex: "[1]"
      sql_hint_algorithm:
        type: SQL_HINT

[TherChenYang]

After verification, the issue can be resolved by adding the following configuration. I will close this issue.

  - !SINGLE
    tables:
      - "*.*"

[bobbyz007]

@zhaojinchao95 @TherChenYang It works, Thx.