HADOOP-16801: S3Guard-listFiles will not query s3 if all listings are… #1815

mustafaiman · 2020-01-22T01:49:53Z

… authoritative

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

hadoop-yetus · 2020-01-23T00:59:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	18m 28s	trunk passed
+1 💚	compile	0m 34s	trunk passed
+1 💚	checkstyle	0m 26s	trunk passed
+1 💚	mvnsite	0m 40s	trunk passed
+1 💚	shadedclient	13m 36s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 29s	trunk passed
+0 🆗	spotbugs	0m 58s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 57s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 34s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
-0 ⚠️	checkstyle	0m 19s	hadoop-tools/hadoop-aws: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)
+1 💚	mvnsite	0m 32s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	16m 8s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 27s	the patch passed
+1 💚	findbugs	1m 8s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 28s	hadoop-aws in the patch passed.
+1 💚	asflicense	0m 30s	The patch does not generate ASF License warnings.
		58m 42s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/3/artifact/out/Dockerfile
GITHUB PR	#1815
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux e099a8c88414 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 9520b2a
Default Java	1.8.0_232
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/3/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/3/testReport/
Max. process+thread count	412 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

steveloughran

thanks for this, it's going to take some careful work from both of us to merge in, but at a quick glance it makes sense.

It's going to have to wait for #1810, which is changing auth mode marking on dirs, and fixes a major problem which has surfaced. The changes will mean tweaks to your test, but they don't step near the production code you've been editing.

made some minor comments as I briefly looked at the initial PR

Reminder: we need to know which S3 endpoint you ran against. We all have to live by that rule.

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/S3Guard.java

.../test/java/org/apache/hadoop/fs/s3a/s3guard/ITestDynamoDBMetadataStoreAuthoritativeMode.java

mustafaiman · 2020-01-23T21:01:33Z

Sorry, I had mentioned the testing in the jira ticket, forgot to add here. tested with s3 us-east-1. -scale -s3guard -dynamo (both -auth and no auth)

mustafaiman · 2020-01-23T23:36:39Z

I re-ran only the two tests as I only made changes to them in the subsequent commit. I used s3 us-west-2.

hadoop-yetus · 2020-01-24T00:56:55Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	18m 50s	trunk passed
+1 💚	compile	0m 34s	trunk passed
+1 💚	checkstyle	0m 27s	trunk passed
+1 💚	mvnsite	0m 38s	trunk passed
+1 💚	shadedclient	13m 31s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 29s	trunk passed
+0 🆗	spotbugs	0m 59s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 56s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 33s	the patch passed
+1 💚	compile	0m 29s	the patch passed
+1 💚	javac	0m 29s	the patch passed
-0 ⚠️	checkstyle	0m 19s	hadoop-tools/hadoop-aws: The patch generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15)
+1 💚	mvnsite	0m 31s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 27s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 25s	the patch passed
+1 💚	findbugs	1m 2s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 18s	hadoop-aws in the patch passed.
+1 💚	asflicense	0m 30s	The patch does not generate ASF License warnings.
		55m 59s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/4/artifact/out/Dockerfile
GITHUB PR	#1815
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux da8d62311ddc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6c1fa24`
Default Java	1.8.0_232
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/4/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/4/testReport/
Max. process+thread count	454 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/4/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-01-24T09:00:47Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	20m 48s	trunk passed
+1 💚	compile	0m 35s	trunk passed
+1 💚	checkstyle	0m 27s	trunk passed
+1 💚	mvnsite	0m 40s	trunk passed
+1 💚	shadedclient	15m 16s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 28s	trunk passed
+0 🆗	spotbugs	1m 1s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 58s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 33s	the patch passed
+1 💚	compile	0m 27s	the patch passed
+1 💚	javac	0m 27s	the patch passed
-0 ⚠️	checkstyle	0m 18s	hadoop-tools/hadoop-aws: The patch generated 1 new + 15 unchanged - 0 fixed = 16 total (was 15)
+1 💚	mvnsite	0m 32s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 14s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 26s	the patch passed
+1 💚	findbugs	1m 6s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 27s	hadoop-aws in the patch passed.
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		59m 46s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/5/artifact/out/Dockerfile
GITHUB PR	#1815
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux d125b16f57b9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `978c487`
Default Java	1.8.0_232
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/5/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/5/testReport/
Max. process+thread count	421 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/5/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

steveloughran · 2020-01-26T16:38:38Z

OK, I've merged in my auth code patch to trunk; you will have to apply yours on top

in the ITestDDBAuthMode code there's explicit methods to create auth/nonauth dirs; please use
and make use of S3ATestUtils.MetricDiff to make sure that LIST requests are absolutely not made -that'll be the regression test

mustafaiman · 2020-01-27T18:49:33Z

@steveloughran
about "2.", I am already using MetricDiff to check for "OBJECT_LIST_REQUESTS". Are you talking about something else?

hadoop-yetus · 2020-01-27T20:28:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	26m 32s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	21m 10s	trunk passed
+1 💚	compile	0m 33s	trunk passed
+1 💚	checkstyle	0m 25s	trunk passed
+1 💚	mvnsite	0m 42s	trunk passed
+1 💚	shadedclient	14m 20s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 31s	trunk passed
+0 🆗	spotbugs	1m 8s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	1m 6s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 37s	the patch passed
+1 💚	compile	0m 30s	the patch passed
+1 💚	javac	0m 30s	the patch passed
-0 ⚠️	checkstyle	0m 20s	hadoop-tools/hadoop-aws: The patch generated 1 new + 20 unchanged - 0 fixed = 21 total (was 20)
+1 💚	mvnsite	0m 35s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 12s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 27s	the patch passed
+1 💚	findbugs	1m 10s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 31s	hadoop-aws in the patch passed.
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		86m 45s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/6/artifact/out/Dockerfile
GITHUB PR	#1815
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 4630d4e78bae 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 7f40e66
Default Java	1.8.0_232
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/6/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/6/testReport/
Max. process+thread count	412 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/6/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

mustafaiman · 2020-01-27T22:56:00Z

Re-ran the tests after rebase. Against s3 us-west-1. Options: "-Dscale -Ds3guard -Dauth -Ddynamo". Only one test fails. I checked the trunk branch. The same test fails consistently on trunk branch too. So not related to my changes:

14:41:46 [ERROR] test_120_terasort[directory](org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A)  Time elapsed: 5.802 s  <<< FAILURE!
14:41:46 java.lang.AssertionError: terasort(s3a://cloudera-dev-7hd9am/terasort-directory/sortin, s3a://cloudera-dev-7hd9am/terasort-directory/sortout) failed expected:<0> but was:<1>
14:41:46 	at org.junit.Assert.fail(Assert.java:88)
14:41:46 	at org.junit.Assert.failNotEquals(Assert.java:834)
14:41:46 	at org.junit.Assert.assertEquals(Assert.java:645)
14:41:46 	at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.executeStage(ITestTerasortOnS3A.java:241)
14:41:46 	at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.test_120_terasort(ITestTerasortOnS3A.java:291)
14:41:46 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14:41:46 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
14:41:46 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
14:41:46 	at java.lang.reflect.Method.invoke(Method.java:498)
14:41:46 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
14:41:46 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
14:41:46 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
14:41:46 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
14:41:46 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
14:41:46 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
14:41:46 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
14:41:46 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
14:41:46 	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
14:41:46 	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
14:41:46 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
14:41:46 	at java.lang.Thread.run(Thread.java:748)
14:41:46 
14:41:46 [ERROR] test_120_terasort[magic](org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A)  Time elapsed: 6.705 s  <<< FAILURE!
14:41:46 java.lang.AssertionError: terasort(s3a://cloudera-dev-7hd9am/terasort-magic/sortin, s3a://cloudera-dev-7hd9am/terasort-magic/sortout) failed expected:<0> but was:<1>
14:41:46 	at org.junit.Assert.fail(Assert.java:88)
14:41:46 	at org.junit.Assert.failNotEquals(Assert.java:834)
14:41:46 	at org.junit.Assert.assertEquals(Assert.java:645)
14:41:46 	at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.executeStage(ITestTerasortOnS3A.java:241)
14:41:46 	at org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A.test_120_terasort(ITestTerasortOnS3A.java:291)
14:41:46 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14:41:46 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
14:41:46 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
14:41:46 	at java.lang.reflect.Method.invoke(Method.java:498)
14:41:46 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
14:41:46 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
14:41:46 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
14:41:46 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
14:41:46 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
14:41:46 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
14:41:46 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
14:41:46 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
14:41:46 	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
14:41:46 	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
14:41:46 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
14:41:46 	at java.lang.Thread.run(Thread.java:748)

bgaborg

Thanks for working on this @mustafaiman.
I can see the direction it's going, but I found some things that we need to discuss first.

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OperationCallbacks.java

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java

hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/s3guard/ImportOperation.java

… authoritative

hadoop-yetus · 2020-01-28T22:56:10Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	18m 6s	trunk passed
+1 💚	compile	0m 34s	trunk passed
+1 💚	checkstyle	0m 27s	trunk passed
+1 💚	mvnsite	0m 39s	trunk passed
+1 💚	shadedclient	13m 28s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 29s	trunk passed
+0 🆗	spotbugs	0m 58s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 56s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 32s	the patch passed
+1 💚	compile	0m 27s	the patch passed
+1 💚	javac	0m 27s	the patch passed
-0 ⚠️	checkstyle	0m 19s	hadoop-tools/hadoop-aws: The patch generated 2 new + 19 unchanged - 0 fixed = 21 total (was 19)
+1 💚	mvnsite	0m 32s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 20s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 26s	the patch passed
+1 💚	findbugs	1m 1s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 28s	hadoop-aws in the patch passed.
+1 💚	asflicense	0m 32s	The patch does not generate ASF License warnings.
		55m 14s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/7/artifact/out/Dockerfile
GITHUB PR	#1815
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux ee01c3ad6573 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `1839c46`
Default Java	1.8.0_232
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/7/artifact/out/diff-checkstyle-hadoop-tools_hadoop-aws.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/7/testReport/
Max. process+thread count	422 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1815/7/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

mustafaiman · 2020-01-29T01:54:09Z

re-ran the same tests against s3 us-west-1

bgaborg

LGTM, +1. Tests were running against Ireland.

mustafaiman · 2020-01-29T16:41:09Z

hey @steveloughran do you have any more comments? Can you merge?

steveloughran · 2020-01-30T10:58:39Z

gabor has merged, no comments from me, but I'm going to rebase my ongoing work onto trunk to help validate this. If I see problems in the next few days, I'll raise followup JIRAs. thanks!

… authoritative (apache#1815). Contributed by Mustafa İman.

…istings are authoritative (apache#1815). Contributed by Mustafa İman. (cherry picked from commit 5977360) Change-Id: Ib5c3517b4b62543614078d213749a3c2f77616f8

mustafaiman force-pushed the HADOOP-16801 branch 2 times, most recently from ac9ea61 to 6b742c9 Compare January 22, 2020 23:50

apache deleted a comment from hadoop-yetus Jan 23, 2020

steveloughran requested changes Jan 23, 2020

View reviewed changes

mustafaiman force-pushed the HADOOP-16801 branch from 6b742c9 to 252108a Compare January 23, 2020 23:32

mustafaiman force-pushed the HADOOP-16801 branch from 252108a to f4a8515 Compare January 24, 2020 07:31

mustafaiman force-pushed the HADOOP-16801 branch from f4a8515 to 638cbdd Compare January 27, 2020 18:59

bgaborg requested changes Jan 28, 2020

View reviewed changes

HADOOP-16801: S3Guard-listFiles will not query s3 if all listings are…

e045381

… authoritative

mustafaiman force-pushed the HADOOP-16801 branch from 638cbdd to e045381 Compare January 28, 2020 21:59

bgaborg approved these changes Jan 29, 2020

View reviewed changes

bgaborg merged commit 5977360 into apache:trunk Jan 30, 2020

mustafaiman deleted the HADOOP-16801 branch January 30, 2020 17:53

RogPodge pushed a commit to RogPodge/hadoop that referenced this pull request Mar 25, 2020

HADOOP-16801. S3Guard listFiles will not query S3 if all listings are…

ac572b4

… authoritative (apache#1815). Contributed by Mustafa İman.

HADOOP-16801: S3Guard-listFiles will not query s3 if all listings are… #1815

HADOOP-16801: S3Guard-listFiles will not query s3 if all listings are… #1815

Uh oh!

Conversation

mustafaiman commented Jan 22, 2020

NOTICE

Uh oh!

hadoop-yetus commented Jan 23, 2020

Uh oh!

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mustafaiman commented Jan 23, 2020

Uh oh!

mustafaiman commented Jan 23, 2020

Uh oh!

hadoop-yetus commented Jan 24, 2020

Uh oh!

hadoop-yetus commented Jan 24, 2020

Uh oh!

steveloughran commented Jan 26, 2020

Uh oh!

mustafaiman commented Jan 27, 2020

Uh oh!

hadoop-yetus commented Jan 27, 2020

Uh oh!

mustafaiman commented Jan 27, 2020

Uh oh!

bgaborg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Jan 28, 2020

Uh oh!

mustafaiman commented Jan 29, 2020

Uh oh!

bgaborg left a comment

Choose a reason for hiding this comment

Uh oh!

mustafaiman commented Jan 29, 2020

Uh oh!

steveloughran commented Jan 30, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants