[ISSUE #8920] Refactor SSL context loading process to support multiple protocols dynamic loading #9483

EnableAsync · 2025-06-17T07:11:37Z

Which Issue(s) This PR Fixes

Brief Description

This PR adds functionality to dynamically reload TLS certificates without requiring a server restart. The implementation monitors certificate and key files for changes and automatically updates the TLS configuration when both files are modified.

Key changes:

Added file monitoring for TLS certificate and private key files
Implemented logic to detect simultaneous changes to both files
Added certificate reload mechanism that updates TLS configuration in-place
Eliminated the need for service restarts when renewing certificates

This feature improves system availability by allowing certificate updates with zero downtime, which is particularly valuable for production environments with automatic certificate renewal processes.

How Did You Test This Change?

I've implemented comprehensive tests to validate the dynamic TLS certificate update feature. The tests cover several key aspects:

TlsCertificateManager Tests

Verified the certificate monitoring logic correctly detects changes in certificate files
Confirmed reload is triggered when both certificate and key files are modified
Validated that changes to trust certificates trigger immediate reload
Ensured the listener notification system works properly when certificate files change

gRPC Server TLS Context Tests

Tested successful SSL context reloading for gRPC servers
Verified proper error handling for certificate exceptions during reload
Confirmed error handling for IO exceptions during reload
Validated that reload handlers are properly registered and unregistered
Verified the SSL context is correctly replaced after a reload event

Remoting Server TLS Context Tests

Confirmed the Netty remoting server properly updates its SSL context when triggered
Validated that the TlsCertificateManager correctly notifies all registered listeners
Verified that the SSL context is actually replaced with a new instance
Tested integration between the certificate manager and the remoting server components

These tests ensure the system can update TLS certificates without requiring a service restart, improving system availability during certificate renewals.

- 在 GrpcServer 和 RemotingProtocolServer 中添加文件监视服务，用于监控 TLS 证书和密钥的变化 - 实现证书和密钥变更时重新加载 SSL 上下文的逻辑 - 优化 ProxyAndTlsProtocolNegotiator 中的 SSL 上下文加载过程 - 添加日志记录，方便调试和监控 TLS 相关操作

- 重构 GrpcServer 中的证书监控逻辑，提取到独立的 GrpcCertKeyFileWatchListener 类中 - 优化证书变更处理流程，提高代码可读性和维护性 - 新增 GrpcServerTest 类，为 gRPC服务器和证书监控添加单元测试- 测试覆盖了各种证书变更场景，包括单独变更和组合变更 - 验证了证书变更时 SSLContext 的重新加载和错误处理 Signed-off-by: Async <raisinata@foxmail.com>

Signed-off-by: Async <raisinata@foxmail.com>

- 移除 FileWatchService，改用 TlsCertificateManager 统一管理 TLS证书 - 实现 TlsContextReloadListener 接口，响应 TLS 证书更新 - 优化 GrpcServer 和 RemotingProtocolServer 中的 TLS 证书更新逻辑 - 新增单元测试验证 TLS 证书更新功能 Signed-off-by: Async <raisinata@foxmail.com>

- 重构了多个测试类中的重复代码- 提高了测试的可读性和维护性 - 确保在测试中正确关闭资源 Signed-off-by: Async <raisinata@foxmail.com>

- 移除了不必要的导入项 - 显式导入了所有活动类，提高了代码的可读性和维护性 Signed-off-by: Async <raisinata@foxmail.com>

qianye1001 · 2025-06-17T07:42:57Z

proxy/src/main/java/org/apache/rocketmq/proxy/grpc/GrpcServer.java

+        }
+    }
+
+    static class GrpcTlsReloadHandler implements TlsCertificateManager.TlsContextReloadListener {


Signed-off-by: Async <raisinata@foxmail.com>

lizhimins · 2025-06-18T05:43:52Z

proxy/src/main/java/org/apache/rocketmq/proxy/service/cert/TlsCertificateManager.java

+        }
+    }
+
+    public static TlsCertificateManager getInstance() {


avoid using a singleton? use Dependency Injection instead

lizhimins · 2025-06-18T07:34:52Z

proxy/src/main/java/org/apache/rocketmq/proxy/grpc/ProxyAndTlsProtocolNegotiator.java

-        sslContext = loadSslContext();
+        try {
+            loadSslContext();
+            log.info("SSLContext created for proxy server");


SSLContext / SslContext 需要保持一致，保持日志的严谨性

lizhimins · 2025-06-18T07:36:13Z

proxy/src/main/java/org/apache/rocketmq/proxy/grpc/ProxyAndTlsProtocolNegotiator.java

+        try {
+            loadSslContext();
+            log.info("SSLContext created for proxy server");
+        } catch (IOException | CertificateException e) {


suggest catch throwable

Here I want to crash directly if an exception is thrown during SSLContext initialization

lizhimins · 2025-06-18T07:41:09Z

proxy/src/main/java/org/apache/rocketmq/proxy/grpc/ProxyAndTlsProtocolNegotiator.java

@@ -231,7 +245,7 @@ protected void handleHAProxyTLV(HAProxyTLV tlv, Attributes.Builder builder) {
            return;
        }
        Attributes.Key<String> key = AttributeKeys.valueOf(
-            HAProxyConstants.PROXY_PROTOCOL_TLV_PREFIX + String.format("%02x", tlv.typeByteValue()));
+                HAProxyConstants.PROXY_PROTOCOL_TLV_PREFIX + String.format("%02x", tlv.typeByteValue()));


codecov-commenter · 2025-06-18T08:02:57Z

Codecov Report

Attention: Patch coverage is 13.46154% with 90 lines in your changes missing coverage. Please review.

Project coverage is 47.99%. Comparing base (cfddd12) to head (b5acb36).
Report is 3 commits behind head on develop.

Files with missing lines	Patch %	Lines
...etmq/proxy/service/cert/TlsCertificateManager.java	0.00%	49 Missing ⚠️
...etmq/proxy/grpc/ProxyAndTlsProtocolNegotiator.java	50.00%	12 Missing and 2 partials ⚠️
...ava/org/apache/rocketmq/proxy/grpc/GrpcServer.java	0.00%	13 Missing ⚠️
...ocketmq/proxy/remoting/RemotingProtocolServer.java	0.00%	9 Missing ⚠️
...n/java/org/apache/rocketmq/proxy/ProxyStartup.java	0.00%	4 Missing ⚠️
...mq/proxy/remoting/MultiProtocolRemotingServer.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@              Coverage Diff              @@
##             develop    #9483      +/-   ##
=============================================
- Coverage      48.15%   47.99%   -0.16%     
+ Complexity     12017    11995      -22     
=============================================
  Files           1308     1309       +1     
  Lines          92174    92331     +157     
  Branches       11790    11815      +25     
=============================================
- Hits           44382    44315      -67     
- Misses         42302    42514     +212     
- Partials        5490     5502      +12

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- 新增 CertChangeEvent、CertChangeSource 和 FileCertChangeSource 类 - 实现证书文件变更监听和事件处理机制 - 修改 GrpcServer 以支持证书变更热重载 - 更新相关测试用例 Signed-off-by: Async <raisinata@foxmail.com>

- 在 GrpcServer 类中添加异常堆栈打印，以便更好地调试问题 - 在 ProxyAndTlsProtocolNegotiator 类中修正 SSLContext 初始化错误日志 Signed-off-by: Async <raisinata@foxmail.com>

- 将 FileCertChangeSource 的具体实现细节封装起来 - 使用 CertChangeSource接口替代具体的 FileCertChangeSource 类 - 提高了代码的可维护性和可扩展性 Signed-off-by: Async <raisinata@foxmail.com>

- 添加对证书路径为空的检查 - 优化证书更新事件处理逻辑 - 修复证书更新时的路径检查问题 Signed-off-by: Async <raisinata@foxmail.com>

lizhimins · 2025-06-19T05:44:40Z

proxy/src/test/java/org/apache/rocketmq/proxy/grpc/GrpcServerTest.java

+//import static org.mockito.Mockito.when;
+//
+//@ExtendWith(MockitoExtension.class)
+//public class GrpcServerTest {


为社么注释掉这些了

- 新增 FileCertChangeSource 类，用于监听证书文件变化 - 添加 GrpcServer 和 MultiProtocolRemotingServer 的 TLS 证书重载处理 - 修改 ProxyAndTlsProtocolNegotiator 类，支持从输入流加载 SSL 上下文 - 删除冗余测试类，优化测试用例 Signed-off-by: Async <raisinata@foxmail.com>

- 修复了一些潜在的代码风格问题 Signed-off-by: Async <raisinata@foxmail.com>

- 在 FileCertChangeSourceTest、GrpcServerTest、GrpcSslContextTest、 MultiProtocolTlsHelperTest 和 TlsCertificateManagerTest 五个测试类中添加了 Apache License 2.0 开源许可证的版权信息 - 删除了 GrpcSslContextTest 中的重复代码 Signed-off-by: Async <raisinata@foxmail.com>

- 修改 CertChangeEvent 类，将 values 字段改为单个 value 字段 - 更新 CertChangeSource 接口，使用事件列表替代单个事件 -调整 FileCertChangeSource 实现，适应新的事件处理方式- 修改 TlsCertificateManager 类，支持处理多个证书变更事件 - 更新相关测试用例，以适应新的事件处理逻辑 Signed-off-by: Async <raisinata@foxmail.com>

EnableAsync added 10 commits June 11, 2025 18:57

fix: code format

eba9a65

Signed-off-by: Async <raisinata@foxmail.com>

test: add test cases

5c8ce5c

Signed-off-by: Async <raisinata@foxmail.com>

fix: code format

63cc595

Signed-off-by: Async <raisinata@foxmail.com>

test(proxy): 优化 TLS 相关测试用例

4df607f

- 重构了多个测试类中的重复代码- 提高了测试的可读性和维护性 - 确保在测试中正确关闭资源 Signed-off-by: Async <raisinata@foxmail.com>

refactor(proxy): 优化代码导入结构

4dca0db

- 移除了不必要的导入项 - 显式导入了所有活动类，提高了代码的可读性和维护性 Signed-off-by: Async <raisinata@foxmail.com>

update

ada9a1c

qianye1001 reviewed Jun 17, 2025

View reviewed changes

EnableAsync added 2 commits June 17, 2025 16:01

fix: no static

bfc1d56

Signed-off-by: Async <raisinata@foxmail.com>

fix: add SingletonHolder for TlsCertificateManager

b5acb36

Signed-off-by: Async <raisinata@foxmail.com>

lizhimins reviewed Jun 18, 2025

View reviewed changes

EnableAsync added 4 commits June 18, 2025 20:23

fix(proxy): 修复 gRPC服务的 SSLContext 初始化错误和日志打印问题

2fcbe40

- 在 GrpcServer 类中添加异常堆栈打印，以便更好地调试问题 - 在 ProxyAndTlsProtocolNegotiator 类中修正 SSLContext 初始化错误日志 Signed-off-by: Async <raisinata@foxmail.com>

refactor(proxy): 优化证书变更源的引用

dc2a4c2

- 将 FileCertChangeSource 的具体实现细节封装起来 - 使用 CertChangeSource接口替代具体的 FileCertChangeSource 类 - 提高了代码的可维护性和可扩展性 Signed-off-by: Async <raisinata@foxmail.com>

fix(proxy): 修复文件证书更新逻辑

6fae362

- 添加对证书路径为空的检查 - 优化证书更新事件处理逻辑 - 修复证书更新时的路径检查问题 Signed-off-by: Async <raisinata@foxmail.com>

lizhimins reviewed Jun 19, 2025

View reviewed changes

EnableAsync added 4 commits June 19, 2025 14:06

refactor(remoting): 优化 TlsHelper 类的代码格式- 调整了代码缩进和格式，提高了代码的可读性

ac1c1d6

- 修复了一些潜在的代码风格问题 Signed-off-by: Async <raisinata@foxmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ISSUE #8920] Refactor SSL context loading process to support multiple protocols dynamic loading #9483

[ISSUE #8920] Refactor SSL context loading process to support multiple protocols dynamic loading #9483

Uh oh!

EnableAsync commented Jun 17, 2025 •

edited

Loading

Uh oh!

qianye1001 Jun 17, 2025

Uh oh!

lizhimins Jun 18, 2025

Uh oh!

lizhimins Jun 18, 2025

Uh oh!

lizhimins Jun 18, 2025

Uh oh!

EnableAsync Jun 18, 2025 •

edited

Loading

Uh oh!

lizhimins Jun 18, 2025

Uh oh!

codecov-commenter commented Jun 18, 2025

Uh oh!

lizhimins Jun 19, 2025

Uh oh!

Uh oh!

[ISSUE #8920] Refactor SSL context loading process to support multiple protocols dynamic loading #9483

Are you sure you want to change the base?

[ISSUE #8920] Refactor SSL context loading process to support multiple protocols dynamic loading #9483

Uh oh!

Conversation

EnableAsync commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which Issue(s) This PR Fixes

Brief Description

How Did You Test This Change?

Uh oh!

qianye1001 Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

lizhimins Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

lizhimins Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

lizhimins Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

EnableAsync Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lizhimins Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Jun 18, 2025

Codecov Report

Uh oh!

lizhimins Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

EnableAsync commented Jun 17, 2025 •

edited

Loading

EnableAsync Jun 18, 2025 •

edited

Loading