optimize: increase the cache of configuration values #2611

funky-eyes · 2020-04-24T15:14:42Z

Ⅰ. Describe what this PR did

optimize: increase the cache of configuration values

大并发下性能相对1.2提升可达90%+

SEATA 1.3(#2611) 性能测试报告

方式说明

nacos做配置&注册中心.

对40条数据随机forupdate读取后,update.设置最高并发200,请求数400,每种方式4次测试

Dubbo下测试: 暂无.

SpringCloud下测试: 678ms 431ms 500ms 427ms (全部正常响应)
File配置中心下测试: 358ms 553ms 421ms 387ms (全部正常响应)
本地事务下测试: 191ms 214ms 109ms 154ms (全部正常响应)

SEATA 1.2下测试: 10289ms 13931ms 11118ms 12792ms (70%+的请求接口响应全部超时)

测试环境

硬件环境: CPU: i5-8300
内存: 8G
硬盘：1TB SSD

软件环境:
系统:win10 64

JAVA: 1.8.0_181

Mysql: 5.7.23

Nacos: 1.2.1

测试模块

测试用例

测试工具

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

codecov-io · 2020-04-25T03:42:41Z

Codecov Report

Merging #2611 into develop will decrease coverage by 0.00%.
The diff coverage is 41.66%.

@@              Coverage Diff              @@
##             develop    #2611      +/-   ##
=============================================
- Coverage      50.82%   50.81%   -0.01%     
- Complexity      2815     2822       +7     
=============================================
  Files            558      559       +1     
  Lines          17931    17971      +40     
  Branches        2126     2132       +6     
=============================================
+ Hits            9113     9132      +19     
- Misses          7952     7968      +16     
- Partials         866      871       +5

Impacted Files	Coverage Δ	Complexity Δ
...e/src/main/java/io/seata/config/Configuration.java	`100.00% <ø> (ø)`	`1.00 <0.00> (ø)`
...c/main/java/io/seata/config/ConfigurationKeys.java	`0.00% <ø> (ø)`	`0.00 <0.00> (ø)`
...a/io/seata/discovery/registry/RegistryService.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...ing/annotation/GlobalTransactionalInterceptor.java	`8.94% <0.00%> (+0.14%)`	`2.00 <0.00> (ø)`
...ain/java/io/seata/config/ConfigurationFactory.java	`45.71% <30.76%> (-4.29%)`	`4.00 <0.00> (ø)`
.../seata/config/SeataConfigurationCacheProvider.java	`66.66% <66.66%> (ø)`	`8.00 <8.00> (?)`
...in/java/io/seata/config/AbstractConfiguration.java	`28.57% <100.00%> (+2.64%)`	`7.00 <1.00> (+1.00)`
...c/main/java/io/seata/config/FileConfiguration.java	`34.84% <100.00%> (ø)`	`6.00 <0.00> (ø)`
...ta/spring/annotation/GlobalTransactionScanner.java	`52.12% <100.00%> (ø)`	`17.00 <0.00> (ø)`
...torage/file/store/FileTransactionStoreManager.java	`56.77% <0.00%> (-0.65%)`	`28.00% <0.00%> (-1.00%)`
... and 4 more

…ead is empty

config/seata-config-core/src/main/java/io/seata/config/AbstractConfiguration.java

objcoding

I said my own understanding:

The previous dynamic downgrade is to monitor the subscription through the configuration center and read the updated cache value as needed.
you use caffeine as a cache locally, and then it expires regularly. There is no need to invalidate the cache regularly. The local cache uses monitoring subscriptions to update the cache in real time, and getConfig reads only the local cache.

The getConfig method will always only interact with the local cache. As for when the cache is updated, it will be handed over to the listen and subscribe logic instead of timing failure. If the local cache fails, it will still request the configuration center.

funky-eyes · 2020-04-26T09:39:51Z

I said my own understanding:

The previous dynamic downgrade is to monitor the subscription through the configuration center and read the updated cache value as needed.

you use caffeine as a cache locally, and then it expires regularly. There is no need to invalidate the cache regularly. The local cache uses monitoring subscriptions to update the cache in real time, and getConfig reads only the local cache.

The getConfig method will always only interact with the local cache. As for when the cache is updated, it will be handed over to the listen and subscribe logic instead of timing failure. If the local cache fails, it will still request the configuration center.

ok

objcoding

about code style

objcoding · 2020-06-11T09:33:59Z

config/seata-config-core/src/main/java/io/seata/config/ConfigurationCache.java

+
+    private static final Set<String> LISTENER_KEYS = new HashSet<>();
+
+    private ConcurrentMap<String, HashSet<ConfigurationChangeListener>> configListenersMap =


just hashMap.

objcoding · 2020-06-11T09:35:49Z

config/seata-config-core/src/main/java/io/seata/config/ConfigurationCache.java

+                    if (CollectionUtils.isEmpty(listenerHashSet)) {
+                        listenerHashSet = new HashSet<>();
+                    }
+                    for (int i = 0; i < listeners.length; i++) {


For code elegance may use foreach would be better.

objcoding · 2020-06-11T09:47:03Z

config/seata-config-core/src/main/java/io/seata/config/ConfigurationCache.java

+            }
+            if (null != listeners && listeners.length > 0) {
+                HashSet<ConfigurationChangeListener> listenerHashSet = null;
+                try {


if (null != listeners && listeners.length > 0) { HashSet<ConfigurationChangeListener> listenerHashSet = getInstance().configListenersMap.computeIfAbsent(dataId, k -> new HashSet<>()); if (CollectionUtils.isEmpty(listenerHashSet)) { listenerHashSet = new HashSet<>(); } for (ConfigurationChangeListener listener : listeners) { ConfigurationChangeListener listener = listeners[i]; if (!listenerHashSet.contains(listener)) { listenerHashSet.add(listener); ConfigurationFactory.getInstance().addConfigListener(dataId, listener); } } }

objcoding

LGTM.

funky-eyes · 2020-06-11T14:37:12Z

@slievrly PTAL

slievrly · 2020-06-13T12:25:03Z

config/seata-config-core/src/main/java/io/seata/config/ConfigurationFactory.java

+            if (null != extConfiguration) {
+                configurationCache = ConfigurationCache.getInstance().proxy(configuration);
+            } else {
+                configurationCache = ConfigurationCache.getInstance().proxy(configuration);


check the code logic.

slievrly · 2020-06-13T14:03:28Z

spring/src/main/java/io/seata/spring/annotation/GlobalTransactionalInterceptor.java

-            degradeCheck = Boolean.parseBoolean(event.getNewValue());
-            if (!degradeCheck) {
+        if (ConfigurationKeys.CLIENT_DEGRADE_CHECK.equals(event.getDataId())) {
+            if (!Boolean.parseBoolean(event.getNewValue())) {


why delete disable process logic?

the code has been restored

slievrly · 2020-06-13T14:04:34Z

spring/src/main/java/io/seata/spring/annotation/GlobalTransactionalInterceptor.java

@@ -107,6 +136,10 @@ public Object invoke(final MethodInvocation methodInvocation) throws Throwable {
            final GlobalTransactional globalTransactionalAnnotation =
                getAnnotation(method, targetClass, GlobalTransactional.class);
            final GlobalLock globalLockAnnotation = getAnnotation(method, targetClass, GlobalLock.class);
+            final boolean disable = ConfigurationFactory.getInstance()


Why local variables?

What happens if the disable configuration item does not exist?

What happens if the disable configuration item does not exist?

when not present, the default value will be used

…dd_config_cache

slievrly

LGTM

jsbxyyx

LGTM

yougecn · 2020-07-10T08:04:38Z

期待1.3发布

funky-eyes added 3 commits April 24, 2020 23:10

add config cache

761ad33

Merge branch 'develop' into add_config_cache

b4cddac

real time reading of current value

a7d3548

funky-eyes closed this Apr 25, 2020

funky-eyes reopened this Apr 25, 2020

funky-eyes added 3 commits April 25, 2020 11:43

optimize code structure

75a38d8

read the current value from the configuration center when the cache r…

ed50977

…ead is empty

fix bug

596fb4c

funky-eyes closed this Apr 25, 2020

funky-eyes reopened this Apr 25, 2020

add dcl

4acaafc

funky-eyes closed this Apr 25, 2020

funky-eyes reopened this Apr 25, 2020

fix bug

5297c55

slievrly reviewed Apr 26, 2020

View reviewed changes

config/seata-config-core/src/main/java/io/seata/config/AbstractConfiguration.java Outdated Show resolved Hide resolved

Merge branch 'develop' into add_config_cache

1ad856f

objcoding reviewed Apr 26, 2020

View reviewed changes

processing cache by aspect

357413a

funky-eyes closed this Apr 26, 2020

funky-eyes reopened this Apr 26, 2020

format code

33d8f16

funky-eyes force-pushed the add_config_cache branch from fcfe3ad to 33d8f16 Compare April 26, 2020 07:48

format code

0c82448

funky-eyes closed this Apr 26, 2020

funky-eyes reopened this Apr 26, 2020

slievrly self-requested a review April 26, 2020 09:12

optimize code

806061d

optimize code structure

8d23187

funky-eyes added 2 commits June 6, 2020 00:35

Merge branch 'develop' into add_config_cache

678d3be

optimize code

f6716fc

funky-eyes closed this Jun 6, 2020

funky-eyes reopened this Jun 6, 2020

funky-eyes added 2 commits June 6, 2020 14:42

optimize code

1b9ef30

optimize code

f63461f

funky-eyes closed this Jun 7, 2020

funky-eyes reopened this Jun 7, 2020

bug fix

5f3e8ac

objcoding reviewed Jun 11, 2020

View reviewed changes

optimize code

f1f83df

objcoding approved these changes Jun 11, 2020

View reviewed changes

Merge branch 'develop' into add_config_cache

c38605a

Merge branch 'develop' into add_config_cache

08f79b5

slievrly reviewed Jun 13, 2020

View reviewed changes

bug fix

4b94955

slievrly reviewed Jun 13, 2020

View reviewed changes

jsbxyyx and others added 7 commits June 17, 2020 12:20

Merge branch 'develop' into add_config_cache

6aae26f

Merge branch 'develop' into add_config_cache

bdee0cc

Merge branch 'develop' into add_config_cache

a1fc788

rollback code

11c5815

Merge branch 'add_config_cache' of github.com:a364176773/seata into a…

0a5e337

…dd_config_cache

rollback code

f20658b

formatting code

8725965

funky-eyes requested a review from slievrly June 21, 2020 11:39

slievrly approved these changes Jun 22, 2020

View reviewed changes

jsbxyyx approved these changes Jun 22, 2020

View reviewed changes

jsbxyyx merged commit 73724a9 into apache:develop Jun 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

optimize: increase the cache of configuration values #2611

optimize: increase the cache of configuration values #2611

funky-eyes commented Apr 24, 2020 •

edited

codecov-io commented Apr 25, 2020 •

edited

objcoding left a comment •

edited

funky-eyes commented Apr 26, 2020

objcoding left a comment

objcoding Jun 11, 2020

funky-eyes Jun 11, 2020

objcoding Jun 11, 2020

funky-eyes Jun 11, 2020

objcoding Jun 11, 2020 •

edited

funky-eyes Jun 11, 2020

objcoding left a comment

funky-eyes commented Jun 11, 2020

slievrly Jun 13, 2020

funky-eyes Jun 13, 2020

slievrly Jun 13, 2020

funky-eyes Jun 21, 2020

slievrly Jun 13, 2020

slievrly Jun 13, 2020

funky-eyes Jun 19, 2020

slievrly left a comment

jsbxyyx left a comment

yougecn commented Jul 10, 2020


		private static final Set<String> LISTENER_KEYS = new HashSet<>();

		private ConcurrentMap<String, HashSet<ConfigurationChangeListener>> configListenersMap =

optimize: increase the cache of configuration values #2611

optimize: increase the cache of configuration values #2611

Conversation

funky-eyes commented Apr 24, 2020 • edited

Ⅰ. Describe what this PR did

SEATA 1.3(#2611) 性能测试报告

方式说明

测试环境

测试模块

Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?

Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews

codecov-io commented Apr 25, 2020 • edited

Codecov Report

objcoding left a comment • edited

Choose a reason for hiding this comment

funky-eyes commented Apr 26, 2020

objcoding left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

objcoding Jun 11, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

objcoding left a comment

Choose a reason for hiding this comment

funky-eyes commented Jun 11, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

slievrly left a comment

Choose a reason for hiding this comment

jsbxyyx left a comment

Choose a reason for hiding this comment

yougecn commented Jul 10, 2020

funky-eyes commented Apr 24, 2020 •

edited

codecov-io commented Apr 25, 2020 •

edited

objcoding left a comment •

edited

objcoding Jun 11, 2020 •

edited