Weekly Report of ShardingSphere (2019.4.8 to 2019.4.12)

[Harper0122]

This report is to review recent changes and developments of ShardingSphere.

Basic Data

Basic data shows how the watch, star and fork have changed during the past week.

Watch	Star	Fork
862	7288	2564

This Week's Development

This part is to show new developments of ShardingSphere during the past week.

Feature Development

New Feature Development

1. Added Sharding-Proxy pass-through function (Issue Support Transparent Mode for Sharding-Proxy #2189).
2. Developed Sharding-Proxy's support for storage process (Issue Support Stored Procedure for Sharding-Proxy IF not sharding #2195).
3. Added Type field in the orchestration module to distinguish the registry center type (Pr add type for orchestration register type. #2186).
4. Added official examples of hint route and broadcast table usage.

Feature Optimization & Refactoring

1. Refactored data masking API and corresponding Yaml, SpringBoot and SpringNameSpace accesses (Issue Reconstruct API of EncryptRules #2185).
2. Adjusted SpringBoot-starter NameSpace (Pr for #2187 revise spring-boot-stater configuration api. #2190).
3. Replaced the parsing engine from its second generation to the third in phase (Issue Use 3rd generation parse engine instead of 2nd generation parse engine #2084).
4. Refactored Algorithm upload of sharding algorithms, auto-augment key and data masking through TypeBasedSPI (Pr using typeBasedSPI to refactor algorithm strategy and factory. #2169).
5. Optimized the HintManager cleaning problem when ShardingConnection is closed (Pr for #2164 do not clear hint manager while connection.close. #2165).
6. Optimized the meta-data information of aggregation functions (such as COUNT(DISTINCT id)) + repetition elimination query to increase query enforcement efficiency (Issue To improve performance of aggregation distinct query #2161).
7. Optimized Insert parsing objects and parsing engine to increase the enforcement efficiency of encryption, decryption and column derivation (Issue Improve the efficiency of InsertOptimizeEngine for ShardingDataSource and EncryptDataSource #2167).
8. Optimized Antlr parsing documents for MySQL DCL and basic expressions.
9. Adjusted the import information of official example in ShardingSphere package.
10. Added data masking test examples, 40% of which have been developed.

Bug Fix

1. Fixed the error report problem of encrypting or decrypting empty-value data or Null data stored in the database (Issue Handle null exception of encrypt() #2196).
2. Fixed the problem that error DataSource may be created when SpringBoot is only configured with broadcast table or default table (Pr fix broadcast table configuration error for spring-boot #2177).
3. Fixed the problem that getColumnIndex() may throw exception when Sharding-JDBC is used in Oracle database query (Pr MemoryQueryResult getColumnIndex throw exception when using Oracle database #2152).
4. Fixed the problem of type shift error when Sharding-JDBC is used in Oracle time field query (Pr ShardingResultSet getTimestamp throw ClassCastException when using oracle database #2159).
5. Fixed the problem of error Insert Value() enforcement in single auto-augment key column (Issue supported hibernate insert values ( ) #2130).

Issue

1. Codes: OpenTracingSQLExecutionHook.toStringList NullPointerException #2142, java.io.NotSerializableException occurred when setting parameters into PreparedStatement #2105，java.io.NotSerializableException: java.io.StringReader #2181 performance test of sharding_proxy #2188, how to recovery when xa transaction manager crashed after commit some resou #2160, support for SQL route extension #2158, Integrate mongodb transaction conflict error #2170.
2. Documents: 不能编译，找了半天也没有找到编译方法 #133.
3. Example Can you make the default-data-source not need included in the data-sources？ #155.

ShardingSphere Ecology Construction

Official Website Construction

1. Updated and published to the official website documents about data sharding, using companies, hint route and orchestration.
2. Renewed English documents of Spring Boot, Spring Namespace, Yaml, Java Configuration and Quick Start.
3. Translated and published the weekly report on the official WeChat account.

Community Construction

1. Solved SkyWalking plug-in conflict and tested framework problem together with location.

Next Week's Plan

This part is to show ShardingSphere's plan for the next week.

Feature Development

New Feature Development

1. Develop Sharding-Proxy's support for storage process.
2. Upgrade and shift ShardingProxy's second-level monitoring application service.
3. Optimize DCL Antlr parsing module.
4. Added official examples for data masking usage.

Problem Investigation

1. Solve SkyWalking test example problem and merge relevant plug-in.
2. Investigate the problem that transactions will report error after Mongodb integrates ShardingSphere.
3. Investigate Issue Table doesn't exist when sharding value in different datasource with in expression #2193.
4. Investigate whether the combination of serviceComb and spring aop can be used to remove spring-aop dependency of sharding-transaction-spring.

ShardingSphere Ecology Construction

Official Website Construction

1. Add explanatory documents for using data masking + sharding in the same time.
2. Add the wiki configuration for JD to access ShardingSphere and the security team to use data masking.

Release

1. Examine the third round of release test.
2. Call the community to vote for ShardingSphere release.