Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt to delayed loading #886

Merged
merged 2 commits into from
Nov 2, 2022
Merged

Adapt to delayed loading #886

merged 2 commits into from
Nov 2, 2022

Conversation

wulangcode
Copy link
Member

Fixes #875

Changes proposed in this pull request:

Check mailbox configuration when submitting. Contributor Guide

@codecov
Copy link

codecov bot commented Nov 1, 2022

Codecov Report

Base: 17.88% // Head: 19.07% // Increases project coverage by +1.19% 🎉

Coverage data is based on head (216db96) compared to base (0f2f7a9).
Patch has no changes to coverable lines.

Additional details and impacted files
@@              Coverage Diff              @@
##             develop     #886      +/-   ##
=============================================
+ Coverage      17.88%   19.07%   +1.19%     
- Complexity       397      430      +33     
=============================================
  Files            208      208              
  Lines           4960     4949      -11     
  Branches         470      470              
=============================================
+ Hits             887      944      +57     
+ Misses          3975     3903      -72     
- Partials          98      102       +4     
Impacted Files Coverage Δ
...in/java/cn/hippo4j/common/toolkit/ContentUtil.java 96.66% <ø> (ø)
...n/hippo4j/common/model/ThreadPoolRunStateInfo.java 0.00% <0.00%> (ø)
...ippo4j/core/executor/DynamicThreadPoolWrapper.java 0.00% <0.00%> (ø)
...core/executor/state/AbstractThreadPoolRuntime.java 0.00% <0.00%> (ø)
...cutor/support/AbstractBuildThreadPoolTemplate.java 0.00% <0.00%> (ø)
...support/adpter/DynamicThreadPoolAdapterChoose.java 0.00% <0.00%> (ø)
...core/executor/support/CommonDynamicThreadPool.java
...ovider/CommonDynamicThreadPoolProviderFactory.java 0.00% <0.00%> (ø)
...ugin/manager/DefaultThreadPoolPluginRegistrar.java 100.00% <0.00%> (+14.28%) ⬆️
...plugin/manager/DefaultThreadPoolPluginManager.java 89.65% <0.00%> (+24.13%) ⬆️
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

/**
* Content util.
*/
public class ContentUtil {

public static String getPoolContent(ThreadPoolParameter parameter) {
if (Objects.isNull(parameter))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的判断似乎没有必要,另外 if 必须跟 {}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个算防御一下,回头我把 {} 加上

@@ -103,9 +103,22 @@ public ClientWorker(HttpAgent httpAgent, String identify, ServerHealthCheck serv

class LongPollingRunnable implements Runnable {

private boolean flag;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

变量命名应该见名知意,cacheMapInitEmptyFlag 是不是更好些

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

晚点我改一下


/**
* Application content post processor.
*/
public class ApplicationContentPostProcessor implements ApplicationListener<ApplicationReadyEvent> {
public class ApplicationContentPostProcessor implements ApplicationListener<ContextRefreshedEvent> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请问这里是因为 ContextRefreshedEvent 会触发唤醒懒加载的 Bean 刷新么,还是?可以简要说下为什么选择这个事件

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

无论是否配置了懒加载都会发送一个ContextRefreshedEvent 的事件,所以我改成了这个。
但是也可能context 会 refresh()多次,所以加了判断只执行一次

@Createsequence
Copy link
Collaborator

我大概理解一下,大佬你的想法是不是这样:
1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒;
2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

@wulangcode
Copy link
Member Author

我大概理解一下,大佬你的想法是不是这样: 1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒; 2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

yes

@Createsequence
Copy link
Collaborator

我大概理解一下,大佬你的想法是不是这样: 1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒; 2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

yes

那基于这个想法,我觉得是否进一步简化一下这个流程,就像 SpringMVC 那样,直接让 ClientWorker 等到第一个线程池被注册的时候才添加长轮询呢?

@wulangcode
Copy link
Member Author

我大概理解一下,大佬你的想法是不是这样: 1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒; 2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

yes

那基于这个想法,我觉得是否进一步简化一下这个流程,就像 SpringMVC 那样,直接让 ClientWorker 等到第一个线程池被注册的时候才添加长轮询呢?

这个被注册的动作你是指,延迟注册的线程池还是未延迟注册的线程池?
还是理解为你觉得不为空才进入长轮训,而不应该在长轮训里面wait

@Createsequence
Copy link
Collaborator

Createsequence commented Nov 2, 2022

我大概理解一下,大佬你的想法是不是这样: 1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒; 2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

yes

那基于这个想法,我觉得是否进一步简化一下这个流程,就像 SpringMVC 那样,直接让 ClientWorker 等到第一个线程池被注册的时候才添加长轮询呢?

这个被注册的动作你是指,延迟注册的线程池还是未延迟注册的线程池? 还是理解为你觉得不为空才进入长轮训,而不应该在长轮训里面wait

被注册 这个行为是针对所有容器中定义了的线程池,也就是说不管线程池是否延迟加载,ClientWorker 仅当有线程池被注册到它内部时才启动轮询,因为在没有任何线程池的情况下,长轮询本身是没有意义的,不知我这么理解对不对。

@wulangcode
Copy link
Member Author

我大概理解一下,大佬你的想法是不是这样: 1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒; 2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

yes

那基于这个想法,我觉得是否进一步简化一下这个流程,就像 SpringMVC 那样,直接让 ClientWorker 等到第一个线程池被注册的时候才添加长轮询呢?

这个被注册的动作你是指,延迟注册的线程池还是未延迟注册的线程池? 还是理解为你觉得不为空才进入长轮训,而不应该在长轮训里面wait

被注册 这个行为是针对所有容器中定义了的线程池,也就是说不管线程池是否延迟加载,ClientWorker 仅当有线程池被注册到它内部时才启动轮询,因为在没有任何线程池的情况下,长轮询本身是没有意义的,不知我这么理解对不对。

你的意思就是不在长轮询里面wait,提到进入长轮询那里。
还有,可能分几种情况来讨论可能清晰点:

  1. 全部线程池为延迟加载
  2. 部分线程池为延迟加载,部分为取消延迟加载。

@Createsequence
Copy link
Collaborator

我大概理解一下,大佬你的想法是不是这样: 1、第一个问题:全局懒加载导致 ApplicationReadyEvent 事件没发送,然后启动后 ClientWorker 一直阻塞没被事件监听器,解决方案是换成 ContextRefreshedEvent 保证 ClientWorker 在项目启动后一定被唤醒; 2、第二个问题:长轮询的时候,可能会因为懒加载导致 ClientWorker.cacheMap 为空,因此无法正常启动长轮询任务,解决方案是不管 cacheMap 是否为空都添加长轮询任务,但是如果 cacheMap 为空,则长轮询任务会让执行线程阻塞,一直到通过 ClientWorker 添加线程池的时候;

yes

那基于这个想法,我觉得是否进一步简化一下这个流程,就像 SpringMVC 那样,直接让 ClientWorker 等到第一个线程池被注册的时候才添加长轮询呢?

这个被注册的动作你是指,延迟注册的线程池还是未延迟注册的线程池? 还是理解为你觉得不为空才进入长轮训,而不应该在长轮训里面wait

被注册 这个行为是针对所有容器中定义了的线程池,也就是说不管线程池是否延迟加载,ClientWorker 仅当有线程池被注册到它内部时才启动轮询,因为在没有任何线程池的情况下,长轮询本身是没有意义的,不知我这么理解对不对。

你的意思就是不在长轮询里面wait,提到进入长轮询那里。 还有,可能分几种情况来讨论可能清晰点:

  1. 全部线程池为延迟加载
  2. 部分线程池为延迟加载,部分为取消延迟加载。

是的,就是这个意思。目前来看注册这个操作是基于后处理器在处理 Bean 的时候触发的,按这个思路想的话,应该是可以让轮询任务的触发延迟到第一个动态线程在容器里被初始化:

  • 全部先加载:线程池实例化,也就是 AbstractApplicationContext 预初始化所有 Bean 并后处理的时候,会触发注册操作,同时添加长轮询任务;
  • 部分先加载,部分懒加载,同上,先初始化的那部分在后处理的时候,会触发注册操作,同时添加长轮询任务:
  • 全部懒加载:得到第一个懒加载的 Bean 被初始化的时候,才会在后处理阶段触发注册操作,同时添加长轮询任务;

@@ -21,6 +21,8 @@
import cn.hippo4j.common.model.ThreadPoolParameter;
import cn.hippo4j.common.model.ThreadPoolParameterInfo;

import java.util.Objects;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

删除无用代码

@magestacks
Copy link
Member

@wulangcode @Createsequence 大概爬了下楼,目前的实现方式是没问题的,如果后续有优化或者其它考虑就再新创建讨论吧,这个 pr 下面就不多展开了。 @wulangcode 解决下多余的导包,就可以合并了

@wulangcode
Copy link
Member Author

@wulangcode @Createsequence 大概爬了下楼,目前的实现方式是没问题的,如果后续有优化或者其它考虑就再新创建讨论吧,这个 pr 下面就不多展开了。 @wulangcode 解决下多余的导包,就可以合并了

删除了

@magestacks magestacks merged commit d02e989 into opengoofy:develop Nov 2, 2022
magestacks added a commit that referenced this pull request Nov 6, 2022
* Tentative contributor update action

* fix : supplement unit test for ThreadPoolManager (#805)

* FIX Data truncation: Data too long for column 'tp_id' at row (#738) (#804)

* add BeanUtilTest test case

* add BeanUtilTest test case

* add BeanUtilTest test case

* add ClassUtilTest test case

* add ConfigExecutorTest test case

* Revert "add ConfigExecutorTest test case"

This reverts commit ffc00ff

* add ConfigExecutorTest test case

* add EnvUtilTest test case

* add ConfigExecutorTest test case

* add Md5ConfigUtilTest test case

* Revert "add Md5ConfigUtilTest test case"

This reverts commit ee949ad

* add Md5ConfigUtilTest test case

* Revert "add Md5ConfigUtilTest test case"

This reverts commit ed63891

* add Md5ConfigUtilTest test case

* FIX Data truncation: Data too long for column 'tp_id' at row (#738)

* FIX Data truncation: Data too long for column 'tp_id' at row (#738)

* Mandatory to specify the ip+port registered by the client (#799)

* Update using company

* Front-end code update

* Official document update

* Released version 1.4.0

* Development of version 1.4.3

* Update README.md (#806)

* Database table structure file adjustment

* Optimize the H2 initialization logic (#807)

* Added config mode springboot 1.x adaptation document

* Fix the CheckStyle code format

* JwtTokenManagerTest 补充测试用例 (#813)

* Refactored thread pool monitoring (#816)

* Refactor the server module (#817)

* Refactor the server module (#817)

* Hippo4j console ui migrated to this project (#818)

* Query the Web thread pool list to add the framework logo (#819)

* Fix the exception to timeout type alarm of lark (#820)

* Add English README

* Change of file format

* Added container and tripartite framework thread pool monitoring (#816)

* feat:Early method return (#823)

* Dynamically registered thread pool queue capacity assignment error (#824)

* Adjust thread pool build notification parameter priority (#825)

* feat:The core thread must not be larger than the maximum thread. (#838)

* organize the toolkit, add references and reference sources (#835)

* fix : organize the toolkit, add references and reference sources

* fix : add MemoryUtil for get memory info

* fix : code format adjustment

* Document update

* Code optimization

* docker-startup.sh的mysql配置多个“-”修复 (#840)

* Lease.isExpired()方法判断过期时间去掉duration (#843)

Co-authored-by: weizhangqing <weizhangqing@icloudshield.com>

* Code optimization

* Code optimization (#847)

* feat:Long polling returns the appropriate status code (#848)

* docker packaging issues fixed

* default subscribe to the remote thread pool configuration. (#850)

* feat:default subscribe to the remote thread pool configuration.

* feat:default empty list

* smart h2 storage path. (#857)

* Add codecov

* Add .codecov.yml

* Update codecov

* Supplemental code comments

* Supplemental code comments (#860)

* Add jacoco-maven-plugin

* Update ci.yml

* Update ci.yml

* Update ci.yml

* Skip the jacoco plugin by default

* Add jacoco-maven-plugin (#861)

* Supplemental code comments

* Add jacoco-maven-plugin

* Update ci.yml

* Update ci.yml

* Update ci.yml

* Skip the jacoco plugin by default

* Change shields icon

* Change shields icon

* Add LocalDataChangeEventTest

* Add LocalDataChangeEventTest (#863)

* Update README.md

* Update README-EN.md

* Add lombok (#867)

* Add LocalDataChangeEventTest

* Add lombok.config

* Update lombok.config

* After the dynamic thread pool is modified multiple times, the queue prompt information is lost (#868)

* Official Documentation Update

* Officially added internationalization function (#866)

* Auto alerts (#872)

* Simplified DynamicThreadPool usage

* Simplified DynamicThreadPool usage

* init

* add Auto Alerts

* The official website opens the multi-version function (#864)

* refactor the function extension logic of DynamicThreadPoolExecutor (#815) (#854)

* feat: Add a new thread-pool that supports the registration of callback interfaces

* feat: Add plugins to support the default extensions of DynamicThreadPoolExecutor

* refactor: Deprecate AbstractDynamicExecutorSupport and make DynamicThreadPoolExecutor extend ExtensibleThreadPoolExecutor (#815)

* fix: obtained plugin list may cause thread-safe problems during iteration

* refactor: Make DynamicThreadPoolExecutor support the selection of shutdown mode in destroy

Co-authored-by: 马称 Ma Chen <machen@apache.org>

* Format code (#873)

* Simplified DynamicThreadPool usage

* Simplified DynamicThreadPool usage

* init

* format code

* Optimize code exception information

* Increase unit test code coverage

* feat:Alarm content (#877)

* In version 1.4.3, the DingTalk alarm keyword was modified (#869)

* Add registered company

* test: Add unit test for DynamicThreadPoolExecutor and ExtensibleThreadPoolExecutor (#884)

* refactor: Extract the logic about calculating task execution time to superclass (#879) (#882)

* test: Add benchmark test for TaskTimeRecord (#885)

* Update .codecov.yml

* Field style refactoring and adding method logs (#878)

* Adjust the conditions for nominating core developers

* Client instance registration information code refactoring (#888) (#889)

* Add method and field annotations (#890)

* Supplementary Methods Notes (#891)

* Add method and field annotations

* Supplementary Methods Notes

* Change the thread pool running data assignment

* Change post log location

* doc: Update developer (#893)

* test: Add unit test about plugin manager (#892)

* Update core developer list

* revert Hippo4jBaseSendMessageService implement CommandLineRunner interface; (#895)

notifyConfigs filed will not be init  when use InitializingBean interface

* Adapt to delayed loading (#886)

* feat:Adapt to delayed loading

* feat:Field Rename

* Supplemental code comments (#898)

* Server-side and client-side model of rpc mode call based on netty (#880)

* fix : add toolkit

* feat : Implement rpc calls through netty, implement server side and client side respectively, the underlying network connection and pipeline context mechanism depend on netty(#812)

* fix : Modifying the comment Format (#812)

* fix bug (#899)

* test: add test for plugins (#896)

* test: add test for plugins

* test: Adjust the time precision of test cases

* fix #901 (#902)

* Supplemental code comments (#904)

* Update .codecov.yml

* Update .codecov.yml

* Spring post processor logic refactoring (#874) (#905)

* Format code (#906)

* Update .codecov.yml

* Format code (#907)

* Add DynamicThreadPoolBannerHandlerTest (#908)

* Update .codecov.yml

* Code optimization and logic refactoring (#909)

* Code optimization and logic refactoring

* Code optimization and logic refactoring

* Remove useless modules and refactor code

* Remove kafka modules

* Remove kafka code

* Update basic project information

* Refactor notification alerts and jar dependencies

* Refactor notification alerts and jar dependencies (#911)

* add new model hippo4j-rpc and  Transfer rpc code to hippo4j-rpc (#912)

* fix : add new model hippo4j-rpc (#812)

* fix : fix : Transfer code to hippo4j-rpc (#812)

* fix : Add set multiple ChannelHandler(#812)

* fix : Code format modification

* Update the front-end thread pool monitoring page

* update hippo4j-rpc pom.xml, Change the value of the packaging t… (#913)

* fix : update hippo4j-rpc pom.xml, Change the value of the packaging tag from pom to jar

* fix : update hippo4j-rpc pom.xml, Change the value of the packaging tag from pom to jar

* Support automatic registration of plugins and plugin registrars (#914)

* feat: Support to centralized management of all plugins

* feat: Support automatic registration of plugins and plugin registrars

* Task execution timeout time assignment

* Refactor dynamic thread pool registration variable naming

* Enable Update Contributor action

* Update pom.xml

* Update the list of contributors (#917)

Co-authored-by: hippo4jbot[bot] <wechat202110@163.com>

* Update the list of contributors (#916)

Co-authored-by: hippo4jbot[bot] <wechat202110@163.com>

* Refactor official documentation

* Update team page

* Stop running status collection when the project is closed gracefully (#918)

* Update official website documents

* Released version 1.4.3

Co-authored-by: pizihao <48643103+pizihao@users.noreply.github.com>
Co-authored-by: Gdk666 <763366136@qq.com>
Co-authored-by: shanjianq <49084314+shanjianq@users.noreply.github.com>
Co-authored-by: zoujin001 <45163196+zoujin001@users.noreply.github.com>
Co-authored-by: WuLang <48200100+wulangcode@users.noreply.github.com>
Co-authored-by: Malcolm <33982485+Malcolmli@users.noreply.github.com>
Co-authored-by: jirong <42790011+w-jirong@users.noreply.github.com>
Co-authored-by: weizhangqing <weizhangqing@icloudshield.com>
Co-authored-by: 王杰 <345127857@qq.com>
Co-authored-by: weihubeats <weihubeats@163.com>
Co-authored-by: 黄成兴 <49221670+Createsequence@users.noreply.github.com>
Co-authored-by: baymax55 <35788491+baymax55@users.noreply.github.com>
Co-authored-by: maxisvest <1447829379@qq.com>
Co-authored-by: hippo4jbot[bot] <wechat202110@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

适配线程池延迟初始化
3 participants