Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HADOOP-18918. ITestS3GuardTool fails if SSE/DSSE encryption is used #6165

Merged
merged 3 commits into from
Oct 20, 2023
Merged

HADOOP-18918. ITestS3GuardTool fails if SSE/DSSE encryption is used #6165

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
766c422
HADOOP-18918. ITestS3GuardTool fails if SSE/DSSE encryption is used
virajjasani Oct 10, 2023
1d3be91
addendum
virajjasani Oct 19, 2023
b0e8436
addendum
virajjasani Oct 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/S3ATestUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1471,6 +1471,20 @@ public static void skipIfEncryptionNotSet(Configuration configuration,
}
}

/**
* Skip a test if any encryption algorithm is used.
*
* @param configuration configuration to probe.
*/
public static void skipIfAnyEncryptionSet(Configuration configuration) throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

think we can remove this now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

String bucket = getTestBucketName(configuration);
final EncryptionSecrets secrets = buildEncryptionSecrets(bucket, configuration);
final String encryptionMethod = secrets.getEncryptionMethod().getMethod();
if (!S3AEncryptionMethods.NONE.getMethod().equals(encryptionMethod)) {
skip(encryptionMethod + " encryption is used");
}
}

/**
* Get the input stream statistics of an input stream.
* Raises an exception if the inner stream is not an S3A input stream
Expand Down
2 changes: 2 additions & 0 deletions hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardTool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
import static org.apache.hadoop.fs.s3a.MultipartTestUtils.countUploadsAt;
import static org.apache.hadoop.fs.s3a.MultipartTestUtils.createPartUpload;
import static org.apache.hadoop.fs.s3a.S3ATestUtils.getLandsatCSVFile;
import static org.apache.hadoop.fs.s3a.S3ATestUtils.skipIfAnyEncryptionSet;
import static org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.BucketInfo;
import static org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.E_BAD_STATE;
import static org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.Uploads;
Expand Down Expand Up @@ -71,6 +72,7 @@ public void testLandsatBucketRequireGuarded() throws Throwable {
@Test
public void testLandsatBucketRequireUnencrypted() throws Throwable {
skipIfClientSideEncryption();
skipIfAnyEncryptionSet(getConfiguration());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of skipping, how about removeBaseAndBucketOverrides(getConfiguration(), Constants.S3_ENCRYPTION_ALGORITHM); and you can remove the skipIfClientSideEncryption() too?

Gets better coverage even if encryption settings are used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, it would be better coverage regardless of encryption configured. Tested the changes with and without encryption.
Thanks @ahmarsuhail

run(BucketInfo.NAME,
"-" + BucketInfo.ENCRYPTION_FLAG, "none",
getLandsatCSVFile(getConfiguration()));
Expand Down