Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

luminous: sse-c-fixes #27295

Merged
merged 7 commits into from Apr 2, 2019

Conversation

Projects
None yet
4 participants
@theanalyst
Copy link
Member

commented Apr 1, 2019

http://tracker.ceph.com/issues/39070

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

cbodley and others added some commits Mar 22, 2019

rgw: decrypt filter does not cross multipart boundaries
multipart uploads with sse encrypts each part separately, using an
initialization vector based on the part offset

decryption must respect the same part boundaries, and start each part
with a fresh initialization vector. this means that the decrypt filter
must flush data up to part boundaries before starting the next

Fixes: http://tracker.ceph.com/issues/38700

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit b782902)
luminous modifications: std::make_unique -> ceph::make_unique
rgw_crypt: take care of start & end boundaries when parsing range
As http range header is inclusive of start and end ranges, this otherwise breaks
at ranges close to multipart boundaries

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9c92b56)
rgw_crypt: make decrypt's parts_len protected
This is so that testing classes can set this and use that to test the branches
in fixup_range tests

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 78b37a5)
test_rgw_crypt: BlockCryptNone takes a block_size param
So that block sizes like 4096 can be tested

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 9e13a42)
tests: rgw crypto tests with partlen vector
4 tests are added when either obj_size for an aligned object, and when an object
is not aligned to a boundary (1 B last part), or part_size is not aligned to a
boundary. Comments on boundary case explains the logic on choosing whether to
align to a 4095B boundary or not

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit 1bf59f6)
luminous changes: std::make_unique -> ceph::make_unique

@theanalyst theanalyst requested a review from cbodley Apr 1, 2019

@tchaikov tchaikov added this to the luminous milestone Apr 1, 2019

@cbodley

This comment has been minimized.

Copy link
Contributor

commented Apr 1, 2019

thanks @theanalyst - i see a make check failure, but jenkins.ceph.com isn't responding to see the logs

are you able to run the new s3tests against this branch? it would be nice to verify that, since the teuthology testing won't include those

@yuriw

This comment has been minimized.

Copy link
Contributor

commented Apr 1, 2019

{

ut_get_sink get_sink;
auto nonecrypt = std::make_unique<BlockCryptNone>(4096);

This comment has been minimized.

Copy link
@cbodley

cbodley Apr 1, 2019

Contributor

s/std::/ceph::/ here too

This comment has been minimized.

Copy link
@theanalyst

theanalyst Apr 2, 2019

Author Member

thanks, I had actually tested the version without the last 2 commits much before, updated this and also ran sse-c s3tests against the new branch and seems to work as expected

theanalyst and others added some commits Mar 27, 2019

test_rgw_crypto: add invalid ranges beyond obj boundary
This is just to assert we're not doing an invalid memory access

Signed-off-by: Abhishek Lekshmanan <abhishek@suse.com>
(cherry picked from commit b34a00d)
rgw: fixup_range() clamps offsets to valid part range
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 4b3bac2)

@theanalyst theanalyst force-pushed the theanalyst:luminous-sse-c-backport branch from 23024a5 to 258f3ab Apr 2, 2019

@theanalyst

This comment has been minimized.

Copy link
Member Author

commented Apr 2, 2019

thanks @theanalyst - i see a make check failure, but jenkins.ceph.com isn't responding to see the logs

are you able to run the new s3tests against this branch? it would be nice to verify that, since the teuthology testing won't include those

Yeah, just verified today again, sse_c tests of the master branch with boto/boto3 seems to pass as expected

aws) [abhishekl@d18:~/spells/storage/s3-tests](⎇ ceph-master)$
S3TEST_CONF=s3.conf ./virtualenv/bin/nosetests s3tests_boto3.functional.test_s3 -m 'sse_c' -vv
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... ok
s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_unaligned_multipart_upload ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok
s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok

----------------------------------------------------------------------
Ran 13 tests in 72.290s

OK
(aws) [abhishekl@d18:~/spells/storage/s3-tests](⎇ ceph-master)$
S3TEST_CONF=s3.conf ./virtualenv/bin/nosetests s3tests.functional.test_s3 -m 'sse_c' -vv
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']
s3tests.functional.test_s3.test_encryption_sse_c_method_head ... ok
s3tests.functional.test_s3.test_encryption_sse_c_present ... ok
s3tests.functional.test_s3.test_encryption_sse_c_other_key ... ok
s3tests.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok
s3tests.functional.test_s3.test_encryption_sse_c_no_md5 ... ok
s3tests.functional.test_s3.test_encryption_sse_c_no_key ... ok
s3tests.functional.test_s3.test_encryption_key_no_sse_c ... ok
s3tests.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok
s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok
s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok
s3tests.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok
s3tests.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok

----------------------------------------------------------------------
Ran 12 tests in 25.432s

OK
@cbodley

cbodley approved these changes Apr 2, 2019

@yuriw yuriw merged commit b5cc77a into ceph:luminous Apr 2, 2019

4 checks passed

Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.