Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-1540. Implement addAcl,removeAcl,setAcl,getAcl for Bucket. Contributed by Ajay Kumar. #874

Merged
merged 4 commits into from May 31, 2019

Conversation

ajayydv
Copy link
Contributor

@ajayydv ajayydv commented May 30, 2019

No description provided.

…ibuted by Ajay Kumar.

(cherry picked from commit 7fc904ca01a281365c89638e8badd7c528c225b0)
@ajayydv ajayydv requested review from xiaoyuyao and anuengineer and removed request for xiaoyuyao May 30, 2019 07:21
@ajayydv ajayydv added the ozone label May 30, 2019
@ajayydv ajayydv requested a review from xiaoyuyao May 30, 2019 07:21
@ajayydv ajayydv self-assigned this May 30, 2019
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 31 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 41 Maven dependency ordering for branch
+1 mvninstall 550 trunk passed
+1 compile 279 trunk passed
+1 checkstyle 74 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 844 branch has no errors when building and testing our client artifacts.
+1 javadoc 143 trunk passed
0 spotbugs 288 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 472 trunk passed
_ Patch Compile Tests _
0 mvndep 15 Maven dependency ordering for patch
+1 mvninstall 490 the patch passed
+1 compile 251 the patch passed
+1 javac 251 the patch passed
-0 checkstyle 37 hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 619 patch has no errors when building and testing our client artifacts.
+1 javadoc 142 the patch passed
+1 findbugs 500 the patch passed
_ Other Tests _
-1 unit 151 hadoop-hdds in the patch failed.
-1 unit 1105 hadoop-ozone in the patch failed.
+1 asflicense 61 The patch does not generate ASF License warnings.
5993
Reason Tests
Failed junit tests hadoop.ozone.container.ozoneimpl.TestOzoneContainer
hadoop.ozone.client.rpc.TestFailureHandlingByClient
hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException
hadoop.ozone.client.rpc.TestWatchForCommit
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-874/1/artifact/out/Dockerfile
GITHUB PR #874
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux f26ce480d1f2 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 2b303e9
Default Java 1.8.0_212
checkstyle https://builds.apache.org/job/hadoop-multibranch/job/PR-874/1/artifact/out/diff-checkstyle-hadoop-ozone.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/1/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/1/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-874/1/testReport/
Max. process+thread count 4958 (vs. ulimit of 5500)
modules C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-874/1/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@ajayydv
Copy link
Contributor Author

ajayydv commented May 30, 2019

Test failures are unrelated.

.setAcls(list)
.build();

metadataManager.getBucketTable().put(dbBucketKey, updatedBucket);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local DB put may not work with OM HA. cc: @bharatviswa504 for additional comments on the proper way to commit OM metadata changes via Ratis.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes in HA path, we shall not do table put, we will use cache and double buffer.
For reference, have a look into HDDS-1551.

I have opened HDDS-1618 to use the newly added code for HA/Non-HA. As this is newly implemented OM request, it can follow the new approach, so that we can avoid refactoring again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a todo to handle this. jira is [HDDS-1619].

Copy link
Contributor

@xiaoyuyao xiaoyuyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ajayydv for the patch. Comments are added inline.
Please also add unit tests for the new code. Thanks!

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 31 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 12 Maven dependency ordering for branch
+1 mvninstall 509 trunk passed
+1 compile 262 trunk passed
+1 checkstyle 73 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 842 branch has no errors when building and testing our client artifacts.
+1 javadoc 147 trunk passed
0 spotbugs 304 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 498 trunk passed
_ Patch Compile Tests _
0 mvndep 19 Maven dependency ordering for patch
+1 mvninstall 502 the patch passed
+1 compile 265 the patch passed
+1 javac 265 the patch passed
+1 checkstyle 72 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 639 patch has no errors when building and testing our client artifacts.
+1 javadoc 141 the patch passed
+1 findbugs 527 the patch passed
_ Other Tests _
+1 unit 237 hadoop-hdds in the patch passed.
-1 unit 1060 hadoop-ozone in the patch failed.
+1 asflicense 43 The patch does not generate ASF License warnings.
6041
Reason Tests
Failed junit tests hadoop.ozone.om.TestScmSafeMode
hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException
hadoop.ozone.client.rpc.TestSecureOzoneRpcClient
hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-874/2/artifact/out/Dockerfile
GITHUB PR #874
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 3f1788c89eea 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 30c6dd9
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/2/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-874/2/testReport/
Max. process+thread count 4469 (vs. ulimit of 5500)
modules C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-874/2/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 51 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 11 Maven dependency ordering for branch
+1 mvninstall 507 trunk passed
+1 compile 259 trunk passed
+1 checkstyle 75 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 813 branch has no errors when building and testing our client artifacts.
+1 javadoc 143 trunk passed
0 spotbugs 302 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 494 trunk passed
_ Patch Compile Tests _
0 mvndep 16 Maven dependency ordering for patch
+1 mvninstall 507 the patch passed
+1 compile 284 the patch passed
+1 javac 284 the patch passed
+1 checkstyle 85 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 668 patch has no errors when building and testing our client artifacts.
+1 javadoc 145 the patch passed
+1 findbugs 519 the patch passed
_ Other Tests _
-1 unit 334 hadoop-hdds in the patch failed.
-1 unit 1313 hadoop-ozone in the patch failed.
+1 asflicense 39 The patch does not generate ASF License warnings.
6418
Reason Tests
Failed junit tests hadoop.hdds.scm.container.placement.algorithms.TestSCMContainerPlacementRackAware
hadoop.ozone.client.rpc.TestHybridPipelineOnDatanode
hadoop.ozone.client.rpc.TestOzoneAtRestEncryption
hadoop.hdds.scm.pipeline.TestPipelineClose
hadoop.ozone.client.rpc.TestOzoneRpcClient
hadoop.hdds.scm.pipeline.TestSCMPipelineManager
hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-874/3/artifact/out/Dockerfile
GITHUB PR #874
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 52f230d9b19a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 9122b9b
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/3/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/3/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-874/3/testReport/
Max. process+thread count 3231 (vs. ulimit of 5500)
modules C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-874/3/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 32 Docker mode activated.
_ Prechecks _
+1 dupname 1 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 37 Maven dependency ordering for branch
+1 mvninstall 589 trunk passed
+1 compile 288 trunk passed
+1 checkstyle 89 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 905 branch has no errors when building and testing our client artifacts.
+1 javadoc 167 trunk passed
0 spotbugs 299 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 491 trunk passed
_ Patch Compile Tests _
0 mvndep 38 Maven dependency ordering for patch
+1 mvninstall 502 the patch passed
+1 compile 316 the patch passed
+1 javac 316 the patch passed
+1 checkstyle 90 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 649 patch has no errors when building and testing our client artifacts.
+1 javadoc 178 the patch passed
+1 findbugs 571 the patch passed
_ Other Tests _
-1 unit 146 hadoop-hdds in the patch failed.
-1 unit 1154 hadoop-ozone in the patch failed.
+1 asflicense 61 The patch does not generate ASF License warnings.
6499
Reason Tests
Failed junit tests hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerCommandHandler
hadoop.ozone.client.rpc.TestFailureHandlingByClient
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-874/4/artifact/out/Dockerfile
GITHUB PR #874
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 666edfb3af78 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 9122b9b
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/4/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/4/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-874/4/testReport/
Max. process+thread count 4615 (vs. ulimit of 5500)
modules C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-874/4/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 31 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 13 Maven dependency ordering for branch
+1 mvninstall 547 trunk passed
+1 compile 293 trunk passed
+1 checkstyle 86 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 887 branch has no errors when building and testing our client artifacts.
+1 javadoc 191 trunk passed
0 spotbugs 311 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 500 trunk passed
_ Patch Compile Tests _
0 mvndep 25 Maven dependency ordering for patch
+1 mvninstall 502 the patch passed
+1 compile 295 the patch passed
+1 javac 295 the patch passed
+1 checkstyle 99 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 146 patch has no errors when building and testing our client artifacts.
+1 javadoc 171 the patch passed
+1 findbugs 511 the patch passed
_ Other Tests _
+1 unit 237 hadoop-hdds in the patch passed.
-1 unit 1036 hadoop-ozone in the patch failed.
+1 asflicense 45 The patch does not generate ASF License warnings.
5809
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-874/5/artifact/out/Dockerfile
GITHUB PR #874
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 8dda786bc398 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 9122b9b
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/5/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-874/5/testReport/
Max. process+thread count 4453 (vs. ulimit of 5500)
modules C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-874/5/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@xiaoyuyao xiaoyuyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, pending CI.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 28 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
0 mvndep 29 Maven dependency ordering for branch
+1 mvninstall 573 trunk passed
+1 compile 302 trunk passed
+1 checkstyle 85 trunk passed
+1 mvnsite 0 trunk passed
+1 shadedclient 974 branch has no errors when building and testing our client artifacts.
+1 javadoc 169 trunk passed
0 spotbugs 347 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 585 trunk passed
_ Patch Compile Tests _
0 mvndep 20 Maven dependency ordering for patch
+1 mvninstall 539 the patch passed
+1 compile 279 the patch passed
+1 javac 279 the patch passed
+1 checkstyle 81 the patch passed
+1 mvnsite 0 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 730 patch has no errors when building and testing our client artifacts.
+1 javadoc 153 the patch passed
+1 findbugs 523 the patch passed
_ Other Tests _
-1 unit 240 hadoop-hdds in the patch failed.
-1 unit 1195 hadoop-ozone in the patch failed.
+1 asflicense 49 The patch does not generate ASF License warnings.
6736
Reason Tests
Failed junit tests hadoop.hdds.scm.container.placement.algorithms.TestSCMContainerPlacementRackAware
hadoop.ozone.om.TestOzoneManagerHA
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-874/6/artifact/out/Dockerfile
GITHUB PR #874
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 2291ebe574ac 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 35f1014
Default Java 1.8.0_212
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/6/artifact/out/patch-unit-hadoop-hdds.txt
unit https://builds.apache.org/job/hadoop-multibranch/job/PR-874/6/artifact/out/patch-unit-hadoop-ozone.txt
Test Results https://builds.apache.org/job/hadoop-multibranch/job/PR-874/6/testReport/
Max. process+thread count 5160 (vs. ulimit of 5500)
modules C: hadoop-ozone/ozone-manager hadoop-ozone/integration-test U: hadoop-ozone
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-874/6/console
versions git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@xiaoyuyao xiaoyuyao merged commit c1d2d92 into apache:trunk May 31, 2019
shanthoosh added a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
SAMZA-2058: Integrate the input stream expansion aware SystemStreamGrouper to JobModel generation flow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
4 participants