Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #297 #302

Merged
Merged

Conversation

tjcelaya
Copy link
Contributor

@tjcelaya tjcelaya commented Aug 5, 2017

Created a unit test for the situation where EncryptingPartEntity finalizes encryption. Not passing yet.

(issue #297)

@tjcelaya tjcelaya self-assigned this Aug 5, 2017
@tjcelaya
Copy link
Contributor Author

tjcelaya commented Aug 8, 2017

Summary of changes:

  • Modify EncryptingEntityHelper#makeCipherOutputForStream to accept SupportedCipherDetails and Cipher objects instead of EncryptionContext so we can pass in cloned Ciphers
  • update makeCipherOutputForStream callers to comply with the aforementioned change
  • copy buffer content into new MultipartOutputStream. Cleanup led to the discovery that this part is unnecessary since the buffer is always empty between parts, can someone validate this?
  • replace the cipherStream field of EncryptionState completely instead of drilling into private state
  • include boolean lastPartAuthWritten in EncryptionStateSnapshot

@tjcelaya
Copy link
Contributor Author

tjcelaya commented Aug 10, 2017

After reviewing with @dekobon the following changes have been requested:

  • Extract wrapped.uploadPart call with surrounding EncryptionStateRecorder usage
  • follow up with @cburroughs about MultipartOutputStream's buffer and keeping the copy code (block comment was added)
  • use .equals for uploadId comparison in EncryptionStateSnapshot

decryptingStream.close();
AssertJUnit.assertArrayEquals(plaintext, decrypted.toByteArray());
}

}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:newline

@@ -183,7 +183,7 @@ public EncryptionState read(final Kryo kryo, final Input input, final Class<Encr
final HMac hmac = kryo.readObjectOrNull(input, HMac.class);

final OutputStream cipherStream = EncryptingEntityHelper.makeCipherOutputForStream(
multipartStream, encryptionContext, hmac);
multipartStream, encryptionContext.getCipherDetails(), encryptionContext.getCipher(), hmac);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does changing the class have any compatibility consequences? I don't think we have a clear version compatibility statement for the kyro module.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that the java-manta-client and java-manta-client-kryo-serialization must be the same version. We should probably document this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a compatibility method and a note to the installation section in the README

@@ -83,12 +88,15 @@ public static OutputStream makeCipherOutputForStream(
* wrapped requires changes to EncryptionStateRecorder!
*
* @param httpOut output stream for writing to the HTTP network socket
* @param encryptionContext current encryption running state
* @param cipherDetails current encryption running state's cipher details
* @param cipher current encryption running state's cipher
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure the phrase "current encryption running state's" will make sense to future readers with EncryptionContext no longer being in the signature.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll clarify this.

final OutputStream httpOut, final EncryptionContext encryptionContext,
final OutputStream httpOut,
final SupportedCipherDetails cipherDetails,
final Cipher cipher,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should cipher be validated for nullness?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems pretty silly to pass a null Cipher to this method, I'll add a call to Validate#notNull

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker for this PR but do we want to validate that the hmac is not null when the CipherDetails indicates the cipher is not AEAD?

* @throws IOException in case of network errors, though MantaMultipartException will be thrown in
* case of invalid response code
*/
private MantaMultipartUploadPart uploadPartSafely(final EncryptedMultipartUpload<WRAPPED_UPLOAD> upload,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: I know it's private, but someone is going to skim the javadoc and be alarmed that the other methods appear "unsafe." Maybe uploadPartWithSnapshot?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why that name didn't pop into my head last night, I was too tired. Making this change.

MantaMultipartUploadPart finalPart = wrapped.uploadPart(upload.getWrapped(),
encryptionState.getLastPartNumber() + 1,
remainderStream.toByteArray());
final MantaMultipartUploadPart finalPart = uploadPartSafely(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this method most always be called inside the encryptionState.getLock(), I think that should get a comment to guard against errant future refactoring.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this change.

return part;
} catch (Exception e) {
if (encryptionState.getLastPartNumber() != partNumber) {
// didn't make it to encryptionState.setLastPartNumber(partNumber)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is setLastPartNumber the official "point of no return"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup

*/
private final HMac hmac;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed because the HMAC will be within the cloned hmaOutputStream, correct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup

return lastPartNumber == that.lastPartNumber
&& cipher == that.cipher
&& hmac == that.hmac;
return Objects.equals(uploadId, that.uploadId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if I breezed by on the last patch, why does this class needs equals & hashcode methods.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're not strictly necessary but it was a matter of habit. The fact that EncryptionState has those methods was the primary motivation.

@cburroughs
Copy link
Contributor

Sorry for the delay, how the questions help.

@tjcelaya tjcelaya merged commit 04ebb25 into TritonDataCenter:master Aug 11, 2017
@tjcelaya tjcelaya deleted the fix/297-MPU-last-part-rewind branch August 11, 2017 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants