[ISSUE #2986] Support for multiple ACL files in a fixed directory #3761

sunxi92 · 2022-01-14T16:13:18Z

Make sure set the target branch to develop

What is the purpose of the change

Support for multiple ACL files in a fixed directory
#2986

Brief changelog

XX

Verifying this change

XXXX

Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test to make sure integration-test pass.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

coveralls · 2022-01-14T16:38:08Z

Coverage increased (+0.5%) to 51.399% when pulling 4dae014 on sunxi92:acl_update into 6dff04f on apache:develop.

codecov-commenter · 2022-01-14T16:38:28Z

Codecov Report

Merging #3761 (a5691ab) into develop (6dff04f) will increase coverage by 0.52%.
The diff coverage is 63.15%.

@@              Coverage Diff              @@
##             develop    #3761      +/-   ##
=============================================
+ Coverage      46.85%   47.37%   +0.52%     
- Complexity      4835     4930      +95     
=============================================
  Files            636      636              
  Lines          42251    42650     +399     
  Branches        5523     5593      +70     
=============================================
+ Hits           19795    20204     +409     
+ Misses         19973    19926      -47     
- Partials        2483     2520      +37

Impacted Files	Coverage Δ
...ocketmq/broker/processor/AdminBrokerProcessor.java	`34.34% <0.00%> (-0.04%)`	⬇️
...mq/common/protocol/body/ClusterAclVersionInfo.java	`0.00% <0.00%> (ø)`
...tocol/header/GetBrokerAclConfigResponseHeader.java	`0.00% <0.00%> (ø)`
...and/acl/ClusterAclConfigVersionListSubCommand.java	`20.40% <0.00%> (-1.82%)`	⬇️
...g/apache/rocketmq/client/impl/MQClientAPIImpl.java	`13.93% <16.66%> (-0.05%)`	⬇️
...pache/rocketmq/acl/plain/PlainAccessValidator.java	`89.70% <50.00%> (-2.61%)`	⬇️
...che/rocketmq/acl/plain/PlainPermissionManager.java	`70.89% <69.86%> (-0.98%)`	⬇️
...tmq/broker/longpolling/PullRequestHoldService.java	`12.04% <0.00%> (-8.44%)`	⬇️
...ocketmq/store/schedule/ScheduleMessageService.java	`66.75% <0.00%> (-7.09%)`	⬇️
...ketmq/common/protocol/body/ConsumerConnection.java	`95.83% <0.00%> (-4.17%)`	⬇️
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6dff04f...a5691ab. Read the comment docs.

caigy

In this PR, there is a very important assumption: The ACL directory MUST contain and ONLY contain ACL config files without subdirectories, but I couldn't find any strong statements and checking about it.
It requires more considerations on compatibility to the original one ACL config file and scalability of supporting multiple config files in different directories.
At present, there are some incompatible modifications:

Return type of AccessValidator.getAclConfigVersion() changed from String to Map<String, DataVersion>, users who have implemented their own implementation would meet NoSuchMethod error.
PlainPermissionManager.getAclConfigDataVersion() is deleted.
Some modification breaks protocol, such as changing the type of version in GetBrokerAclConfigResponseHeader, removing aclConfigDataVersion in ClusterAclVersionInfo.
Users using DEFAULT_PLAIN_ACL_FILE (/conf/plain_acl.yml) can't load ACL config file after this change, because it will be loaded from /conf/acl directory. If they change the acl directory to /conf/, not only acl config files, but also other config files like broker config will be loaded, and the loading process will encounter errors.

caigy · 2022-01-18T05:46:03Z

acl/src/main/java/org/apache/rocketmq/acl/AccessValidator.java

@@ -60,16 +63,18 @@
     *
     * @return
     */
-    String getAclConfigVersion();
+    Map<String, DataVersion> getAclConfigVersion();


It makes the interface incompatible to older versions, you'd better add a new method and deprecate the original ones.

caigy · 2022-01-18T08:01:16Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

-    private static final String DEFAULT_PLAIN_ACL_FILE = "/conf/plain_acl.yml";
-
    private String fileHome = System.getProperty(MixAll.ROCKETMQ_HOME_PROPERTY,
        System.getenv(MixAll.ROCKETMQ_HOME_ENV));

-    private String fileName = System.getProperty("rocketmq.acl.plain.file", DEFAULT_PLAIN_ACL_FILE);


Users using DEFAULT_PLAIN_ACL_FILE can't load ACL config file after this change, it requires more compatible considerations.

+1.
This pr need to supply a detailed design.

It seems users can't specify config file by property rocketmq.acl.plain.file?

caigy · 2022-01-18T08:22:29Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+        if (fileHome == null || fileHome.isEmpty()) {
+            throw new AclException(String.format("%s file is empty", fileHome));
+        }


Is it more appropriate to just returning than throwing exception?

caigy · 2022-01-18T08:37:51Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+        File aclDir = new File(defaultAclDir);
+        File[] aclFiles = aclDir.listFiles();


Array of File objects is created here but only path string is used(org.apache.rocketmq.acl.common.AclUtils#getYamlDataObject uses file name to read this file again), it will use less resource if using java.nio.file.Files#list, or add overloaded method for org.apache.rocketmq.acl.common.AclUtils#getYamlDataObject accepting file object.

caigy · 2022-01-18T08:44:06Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+        List<String> fileList = new ArrayList<>();
+        for (File aclFile : aclFiles) {
+            String aclFileAbsolutePath = aclFile.getAbsolutePath();
+            load(aclFileAbsolutePath);


Should we surround load by try-catch?

caigy · 2022-01-18T11:50:03Z

...c/main/java/org/apache/rocketmq/common/protocol/header/GetBrokerAclConfigResponseHeader.java

 public class GetBrokerAclConfigResponseHeader implements CommandCustomHeader {

    @CFNotNull
-    private String version;
+    private Map<String, DataVersion> version;


It changes the protocol, some users may already use it as String.

caigy · 2022-01-18T11:56:55Z

common/src/main/java/org/apache/rocketmq/common/protocol/body/ClusterAclVersionInfo.java

 public class ClusterAclVersionInfo extends RemotingSerializable {

    private String brokerName;

    private String brokerAddr;

-    private DataVersion aclConfigDataVersion;


Add a new field other than changing it for compatibility.

caigy · 2022-01-18T12:09:45Z

srvutil/src/main/java/org/apache/rocketmq/srvutil/AclFileWatchService.java

+                } else {
+                    for (int i = 0; i < aclFilesNum; i++) {
+                        String fileName = aclFiles[i].getAbsolutePath();
+                        String newHash = hash(fileName);


Calculating hash of a file consumes a lot of system resources, can file modified time and file size be used instead?

caigy · 2022-01-18T12:13:19Z

srvutil/src/main/java/org/apache/rocketmq/srvutil/AclFileWatchService.java

+                    for (int i = 0; i < aclFilesNum; i++) {
+                        String fileName = aclFiles[i].getAbsolutePath();
+                        String newHash = hash(fileName);
+                        if (!newHash.equals(fileCurrentHash.get(i))) {


Should it be fileCurrentHash.get(fileName)?

caigy · 2022-01-18T12:14:53Z

srvutil/src/main/java/org/apache/rocketmq/srvutil/AclFileWatchService.java

+    private int aclFilesNum;
+    private final Map<String, String> fileCurrentHash;
+    private final AclFileWatchService.Listener listener;
+    private static final int WATCH_INTERVAL = 500;


Is it too often?

dongeforever · 2022-01-21T03:53:11Z

The multi ACL files are not compatible with the default ACL file naturally.
If it is necessary to support multi ACL files, a detailed design with compatibility is needed.

dongeforever · 2022-01-21T04:01:28Z

IMO.
The default access validator should keep simple. If it has difficulty supporting the requirements, it is better to supply a new validator implemntation, instead of changing the default one.

…le and scalability of supporting multiple config files in different directories.

caigy · 2022-01-25T02:23:30Z

@sunxi92 Could you post your detailed design so that more people can join this pr? BTW, it should be well documented about how to use this new feature and how it effects the old implementation, not only in comments but also in user guide.

caigy · 2022-01-25T03:06:02Z

...c/main/java/org/apache/rocketmq/tools/command/acl/ClusterAclConfigVersionListSubCommand.java

-            versionNum,
-            timeStampStr
-        );
+        for (Map.Entry<String, DataVersion> entry : aclDataVersion.entrySet()) {


Would AccessValidator.getAllAclConfigVersion() return an empty set?

caigy · 2022-01-25T03:23:17Z

acl/src/main/java/org/apache/rocketmq/acl/AccessValidator.java

+    @Deprecated
    String getAclConfigVersion();


Pls add more description about:

the reason to deprecate String getAclConfigVersion() and replace it with Map<String, DataVersion> getAllAclConfigVersion()

What is the object or scope that version describe? In previous implementation, version is used to mark the whole config. In this multi-filed feature, each config file has its own version. And if ACL config are stored in DB, each record may have its own version. We should define version precisely.

In Map<String, DataVersion> getAllAclConfigVersion(), what is the key of Map? Users may curious of it. If this is the full path of config files, it brings implementation to interface.

caigy · 2022-01-25T03:28:24Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

-    private static final String DEFAULT_PLAIN_ACL_FILE = "/conf/plain_acl.yml";
-
    private String fileHome = System.getProperty(MixAll.ROCKETMQ_HOME_PROPERTY,
        System.getenv(MixAll.ROCKETMQ_HOME_ENV));

-    private String fileName = System.getProperty("rocketmq.acl.plain.file", DEFAULT_PLAIN_ACL_FILE);


It seems users can't specify config file by property rocketmq.acl.plain.file?

caigy · 2022-01-25T03:30:20Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java


-    private Map<String/** AccessKey **/, PlainAccessResource> plainAccessResourceMap = new HashMap<>();
+    private Map<String/** aclFileName **/, Map<String/** AccessKey **/, PlainAccessResource>> aclPlainAccessResourceMap = new HashMap<>();


Is 'file full path' more precise than 'aclFileName'?

caigy · 2022-01-25T03:49:32Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+            File f = new File(fileName);
+            if (fileName.endsWith(".yml")) {
+                fileList.add(fileName);
+            } else if (!f.isFile()) {


Maybe it should be file.isDirectory()?

caigy · 2022-01-25T05:06:23Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+                this.globalWhiteRemoteAddressStrategy.remove(remoteAddressStrategyList.get(i));
+            }
+            this.globalWhiteRemoteAddressStrategyMap.remove(aclFilePath);


Removing items in globalWhiteRemoteAddressStrategy may cause ACL check failures.

caigy · 2022-01-25T05:06:52Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+        if (dataVersionMap.size() == 1) {
+            for (Map.Entry<String, DataVersion> entry : dataVersionMap.entrySet()) {
+                dataVersion.assignNewOne(entry.getValue());
+            }
+        }


Would dataVersion be empty or updated if there are multiple ACL config files?

caigy · 2022-01-25T05:07:18Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+        if (aclPlainAccessResourceMap.size() != 0 && accessKeyTable.size() != 0) {
+            aclPlainAccessResourceMap.clear();
+            accessKeyTable.clear();
+        }
+        if (globalWhiteRemoteAddressStrategy.size() != 0) {
+            globalWhiteRemoteAddressStrategy.clear();
+        }
+        if (globalWhiteRemoteAddressStrategyMap.size() != 0) {
+            globalWhiteRemoteAddressStrategyMap.clear();
+        }


Clearing globalWhiteRemoteAddressStrategy or globalWhiteRemoteAddressStrategyMap may cause ACL check failures before load() finished.

caigy · 2022-01-26T02:51:15Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+            globalWhiteRemoteAddrList.clear();
+            if (globalWhiteAddrsList != null) {
+                globalWhiteRemoteAddrList.addAll(globalWhiteAddrsList);


Combination of clear() and addAll() is not atomic, which may cause ACL failure in this process.

caigy · 2022-01-26T03:02:24Z

...c/main/java/org/apache/rocketmq/common/protocol/header/GetBrokerAclConfigResponseHeader.java

    private String version;

+    private Map<String, DataVersion> versions;


version and versions are too similar and confusing, maybe we should name them more clearly.

2.add a detailed design document

lizhiboo · 2022-01-18T12:06:54Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java


 public class PlainPermissionManager {

    private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.COMMON_LOGGER_NAME);

-    private static final String DEFAULT_PLAIN_ACL_FILE = "/conf/plain_acl.yml";
-


remove old acl file directory incompatible with old version. Can old acl file and new acl dir coexist?

Default acl file is added.

lizhiboo · 2022-02-09T02:41:18Z

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

+    public List<String> getAllAclFiles(String path) {
+        List<String>  allAclFileFullPath = new ArrayList<>();
+        File file = new File(path);
+        File[] files = file.listFiles();


IMO, check file is directory will be better

acl/src/main/java/org/apache/rocketmq/acl/plain/PlainPermissionManager.java

caigy · 2022-02-14T02:28:55Z

srvutil/src/main/java/org/apache/rocketmq/srvutil/AclFileWatchService.java


    public AclFileWatchService(String path, final AclFileWatchService.Listener listener) throws Exception {
        this.aclPath = path;
+        this.defaultAclFile = path.substring(0, path.length() - 4) + System.getProperty("rocketmq.acl.plain.file", "conf/plain_acl.yml");


Just get parent directory instead of string truncation, or make path as a List.

2.Add the logic to check if path is a directory in the method of getAllAclFiles(String path)

…th) method in PlainPermissionManager.java and AclFileWatchService.java

…lConfigVersion command

…AclConfigVersion command 2.Improve the logic of updateAccessConfig method

lizhiboo · 2022-02-17T06:34:48Z

client/src/main/java/org/apache/rocketmq/client/impl/MQClientAPIImpl.java

@@ -387,6 +388,12 @@ public ClusterAclVersionInfo getBrokerClusterAclInfo(final String addr,
                clusterAclVersionInfo.setBrokerName(responseHeader.getBrokerName());
                clusterAclVersionInfo.setBrokerAddr(responseHeader.getBrokerAddr());
                clusterAclVersionInfo.setAclConfigDataVersion(DataVersion.fromJson(responseHeader.getVersion(), DataVersion.class));
+                HashMap<String, Object> dataVersionMap = JSON.parseObject(responseHeader.getAllAclFileVersion(), HashMap.class);
+                Map<String, DataVersion> allAclConfigDataVersion = new HashMap<>();


IMO, specify the concrete type for HashMap for java 1.6.

…ry (apache#3761) * acl temp * acl * fix test case * fix code style issues * add considerations on compatibility to the original one ACL config file and scalability of supporting multiple config files in different directories. * fix test case testWatch * 1.fix some issues 2.add a detailed design document * Add warn log when the accesskey is repeated in multiple ACL files. * 1.Change the folder of acl configuration to conf/acl 2.Add the logic to check if path is a directory in the method of getAllAclFiles(String path) * Add a parameter in AclFileWatchService constructor. * Add logic to determine if path exists in the getAllAclFiles(String path) method in PlainPermissionManager.java and AclFileWatchService.java * Fix the serialization problem of allAclFileVersion field in clusterAclConfigVersion command * 1.Fix the serialization problem of allAclFileVersion field in clusterAclConfigVersion command 2.Improve the logic of updateAccessConfig method

sunxi92 added 4 commits January 14, 2022 23:37

acl temp

b8640b9

acl

645d35a

fix test case

b56b237

fix code style issues

2e848e9

sunxi92 mentioned this pull request Jan 14, 2022

[ISSUE #2986] Support for multiple ACL files in a fixed directory #3090

Closed

duhenglucky requested a review from zongtanghu January 15, 2022 09:28

duhenglucky added this to the 4.9.3 milestone Jan 17, 2022

duhenglucky added module/acl type/enhancement labels Jan 17, 2022

caigy requested changes Jan 18, 2022

View reviewed changes

sunxi92 added 2 commits January 23, 2022 13:22

add considerations on compatibility to the original one ACL config fi…

30f7363

…le and scalability of supporting multiple config files in different directories.

fix test case testWatch

e6e3548

caigy requested changes Jan 26, 2022

View reviewed changes

sunxi92 added 2 commits January 28, 2022 16:28

1.fix some issues

6248b47

2.add a detailed design document

Add warn log when the accesskey is repeated in multiple ACL files.

d4629ac

lizhiboo approved these changes Feb 14, 2022

View reviewed changes

caigy requested changes Feb 14, 2022

View reviewed changes

sunxi92 added 5 commits February 15, 2022 10:04

1.Change the folder of acl configuration to conf/acl

2d877e6

2.Add the logic to check if path is a directory in the method of getAllAclFiles(String path)

Add a parameter in AclFileWatchService constructor.

b323e8e

Add logic to determine if path exists in the getAllAclFiles(String pa…

a5691ab

…th) method in PlainPermissionManager.java and AclFileWatchService.java

Fix the serialization problem of allAclFileVersion field in clusterAc…

05d1207

…lConfigVersion command

1.Fix the serialization problem of allAclFileVersion field in cluster…

4dae014

…AclConfigVersion command 2.Improve the logic of updateAccessConfig method

lizhiboo approved these changes Feb 17, 2022

View reviewed changes

caigy approved these changes Feb 18, 2022

View reviewed changes

duhenglucky merged commit c1bb6aa into apache:develop Feb 19, 2022

		File aclDir = new File(defaultAclDir);
		File[] aclFiles = aclDir.listFiles();


		private Map<String/ AccessKey /, PlainAccessResource> plainAccessResourceMap = new HashMap<>();
		private Map<String/ aclFileName /, Map<String/ AccessKey /, PlainAccessResource>> aclPlainAccessResourceMap = new HashMap<>();

		private String version;

		private Map<String, DataVersion> versions;

[ISSUE #2986] Support for multiple ACL files in a fixed directory #3761

[ISSUE #2986] Support for multiple ACL files in a fixed directory #3761

Conversation

sunxi92 commented Jan 14, 2022

What is the purpose of the change

Brief changelog

Verifying this change

coveralls commented Jan 14, 2022 • edited

codecov-commenter commented Jan 14, 2022 • edited

Codecov Report

caigy left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dongeforever commented Jan 21, 2022

dongeforever commented Jan 21, 2022

caigy commented Jan 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coveralls commented Jan 14, 2022 •

edited

codecov-commenter commented Jan 14, 2022 •

edited

caigy left a comment •

edited