HADOOP-16396. Allow authoritative mode on a subdirectory. #1043

mackrorysd · 2019-07-01T15:29:05Z

No description provided.

hadoop-yetus · 2019-07-01T16:26:25Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	69	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1146	trunk passed
+1	compile	34	trunk passed
+1	checkstyle	22	trunk passed
+1	mvnsite	38	trunk passed
+1	shadedclient	806	branch has no errors when building and testing our client artifacts.
+1	javadoc	24	trunk passed
0	spotbugs	58	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	57	trunk passed
		_ Patch Compile Tests _
-1	mvninstall	28	hadoop-aws in the patch failed.
-1	compile	29	hadoop-aws in the patch failed.
-1	javac	29	hadoop-aws in the patch failed.
-0	checkstyle	18	hadoop-tools/hadoop-aws: The patch generated 12 new + 45 unchanged - 1 fixed = 57 total (was 46)
-1	mvnsite	30	hadoop-aws in the patch failed.
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	868	patch has no errors when building and testing our client artifacts.
+1	javadoc	22	the patch passed
-1	findbugs	33	hadoop-aws in the patch failed.
		_ Other Tests _
-1	unit	35	hadoop-aws in the patch failed.
+1	asflicense	30	The patch does not generate ASF License warnings.
		3381

Subsystem	Report/Notes
Docker	Client=18.09.5 Server=18.09.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/Dockerfile
GITHUB PR	#1043
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux f2567c1d40ce 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `9df6275`
Default Java	1.8.0_212
mvninstall	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-aws.txt
compile	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/patch-compile-hadoop-tools_hadoop-aws.txt
javac	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/patch-compile-hadoop-tools_hadoop-aws.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
mvnsite	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt
findbugs	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/testReport/
Max. process+thread count	344 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-07-01T20:02:40Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	92	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 3 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1171	trunk passed
+1	compile	34	trunk passed
+1	checkstyle	23	trunk passed
+1	mvnsite	39	trunk passed
+1	shadedclient	794	branch has no errors when building and testing our client artifacts.
+1	javadoc	24	trunk passed
0	spotbugs	58	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	55	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	31	the patch passed
+1	compile	29	the patch passed
+1	javac	29	the patch passed
-0	checkstyle	18	hadoop-tools/hadoop-aws: The patch generated 12 new + 45 unchanged - 1 fixed = 57 total (was 46)
+1	mvnsite	33	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	817	patch has no errors when building and testing our client artifacts.
+1	javadoc	21	the patch passed
+1	findbugs	61	the patch passed
		_ Other Tests _
+1	unit	287	hadoop-aws in the patch passed.
+1	asflicense	28	The patch does not generate ASF License warnings.
		3655

Subsystem	Report/Notes
Docker	Client=18.09.5 Server=18.09.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/2/artifact/out/Dockerfile
GITHUB PR	#1043
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 7582216a6d40 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `f8d62a9`
Default Java	1.8.0_212
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/2/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/2/testReport/
Max. process+thread count	333 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1043/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

bgaborg · 2019-07-03T16:17:33Z

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java

@@ -2415,6 +2421,7 @@ void maybeCreateFakeParentDirectory(Path path)
        result.add(files.next());
      }
      // merge the results. This will update the store as needed
+


nit: newline without purpose

bgaborg · 2019-07-03T16:17:55Z

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java

        if (recursive) {
          final PathMetadata pm = metadataStore.get(path, true);
          // shouldn't need to check pm.isDeleted() because that will have
          // been caught by getFileStatus above.
+


nit: newline without purpose

bgaborg · 2019-07-03T16:18:16Z

...p-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/DynamoDBMetadataStore.java

@@ -1321,7 +1321,6 @@ public void put(
      final DirListingMetadata meta,
      @Nullable final BulkOperationState operationState) throws IOException {
    LOG.debug("Saving to table {} in region {}: {}", tableName, region, meta);
-


nit: newline removed without purpose

bgaborg · 2019-07-03T17:43:02Z

+1. tested against ireland. there are some unrelated failures.

(cherry picked from commit 34747c3) Conflicts: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java Change-Id: Id39b3329981fece07d16df7dcab0a8ff109e4f6d

SAMZA-2202: Create log compact topic with larger message size

HADOOP-16396

38cd137

Iterating on Gabor's feedback

3637cf1

mackrorysd force-pushed the HADOOP-16396 branch from e2289d4 to 3637cf1 Compare July 1, 2019 17:16

bgaborg reviewed Jul 3, 2019

View reviewed changes

mackrorysd merged commit 34747c3 into apache:trunk Jul 3, 2019

mackrorysd mentioned this pull request Jul 3, 2019

HADOOP-16409. Allow authoritative mode on non-qualified paths. #1054

Merged

bshashikant pushed a commit to bshashikant/hadoop that referenced this pull request Jul 10, 2019

HADOOP-16396. Allow authoritative mode on a subdirectory. (apache#1043)

397a961

shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019

Merge pull request apache#1043 from lhaiesp/master

f6e0118

SAMZA-2202: Create log compact topic with larger message size

amahussein pushed a commit to amahussein/hadoop that referenced this pull request Oct 29, 2019

HADOOP-16396. Allow authoritative mode on a subdirectory. (apache#1043)

450b721

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-16396. Allow authoritative mode on a subdirectory. #1043

HADOOP-16396. Allow authoritative mode on a subdirectory. #1043

mackrorysd commented Jul 1, 2019

hadoop-yetus commented Jul 1, 2019

hadoop-yetus commented Jul 1, 2019

bgaborg Jul 3, 2019

bgaborg Jul 3, 2019

bgaborg Jul 3, 2019

bgaborg commented Jul 3, 2019

HADOOP-16396. Allow authoritative mode on a subdirectory. #1043

HADOOP-16396. Allow authoritative mode on a subdirectory. #1043

Conversation

mackrorysd commented Jul 1, 2019

hadoop-yetus commented Jul 1, 2019

hadoop-yetus commented Jul 1, 2019

bgaborg Jul 3, 2019

Choose a reason for hiding this comment

bgaborg Jul 3, 2019

Choose a reason for hiding this comment

bgaborg Jul 3, 2019

Choose a reason for hiding this comment

bgaborg commented Jul 3, 2019