NIFI-1769: Implemented SSE with KMS. #1126
Conversation
miquillo
changed the title
There was a problem hiding this comment.
Validating the correct combination of property choices seems awkward and unnecessary in onTrigger, as these property selections are not being evaluated from Expression Language. Instead, how about implementing a customValidate() method to make sure the properties are sensible before the processor may be started? I recommend taking a look at AbstractAWSProcessor::customValidate, both for how it validates and for how it incorporates validation errors from it's super() type.
After validating the properties, I think the onTrigger behavior can be much simpler, maybe just checking to add the key management params.
| @@ -39,47 +80,6 @@ | |||
| import java.util.concurrent.locks.Lock; | |||
| import java.util.concurrent.locks.ReentrantLock; | |||
|
|
|||
| import com.amazonaws.services.s3.model.AmazonS3Exception; | |||
There was a problem hiding this comment.
Would you please only commit changes to the imports, and not re-sort them? I assume your IDE is doing this to "help" you, but it makes the diffs harder to understand.
| .defaultValue(NO_SERVER_SIDE_ENCRYPTION) | ||
| .build(); | ||
|
|
||
| public static final PropertyDescriptor AWS_KMS_KEY = new PropertyDescriptor.Builder() | ||
| .name("aws-kms-key") | ||
| .displayName("Server Side Encryption using KMS") |
There was a problem hiding this comment.
I recommend the displayName be a bit more direct, this field is the alias of a KMS key, and it's going to be used for SSE.
| @@ -50,14 +44,19 @@ | |||
| import org.junit.Ignore; | |||
| import org.junit.Test; | |||
|
|
|||
| import com.amazonaws.AmazonClientException; | |||
There was a problem hiding this comment.
Same thing again about re-sorting imports.
| @@ -458,6 +471,13 @@ public void process(final InputStream rawIn) throws IOException { | |||
| // single part upload | |||
| //---------------------------------------- | |||
| final PutObjectRequest request = new PutObjectRequest(bucket, key, in, objectMetadata); | |||
| if (keyId != null) { | |||
| if (!context.getProperty(SIGNER_OVERRIDE).getValue().equals("AWSS3V4Signer")) { | |||
There was a problem hiding this comment.
Would it be enough to check that it's not V2? I don't think we need to make it impossible to get wrong, as long as we make a good faith attempt to help them get it right. I'm thinking of a few things -
- The default should now be V4, I would prefer we not force users to nail down their signature version
- AWS regions and SDK versions are complicated, for example we don't check if your region supports V4
- If or when AWS comes out with signature V5, we would have to update this field
| if (keyId != null) { | ||
| if (!context.getProperty(SIGNER_OVERRIDE).getValue().equals("AWSS3V4Signer")) { | ||
| getLogger().error("Uploading with AWS:KMS requires S3V4 signature, please enable it"); | ||
| return; |
There was a problem hiding this comment.
The return here routes the flowfile to success without going to S3. Is that intended?
| if (keyId != null) { | ||
| if (!context.getProperty(SIGNER_OVERRIDE).getValue().equals("AWSS3V4Signer")) { | ||
| getLogger().error("Uploading with AWS:KMS requires S3V4 signature, please enable it"); | ||
| return; |
There was a problem hiding this comment.
Same issue with return routing the flowfile to success.
|
@miquillo will you have a chance to address james feedback? I'd like to get PRs which have stalled closed out or completed. Thanks |
|
I'm sorry but I don't see any time in the coming weeks to finish this. We're currently not using this feature anymore, so I propose we close this PR and don't add it to the project. |
|
Thanks for taking a go at this, @miquillo. |
|
This is currently a blocker for us and I have the resources to get this over the line. Please re-open and I will work to ensure the pull request is of a sufficient quality to get merged. |
|
@baank - Thanks for being willing to work on this. A new pull request might be easier. |
No description provided.