rgw: rgwx-skip-decrypt also skips decompression of encrypted objects

[cbodley]

#46188 added support for combining encryption and compression at the same time, but multisite does not replicate these objects correctly. the decompression filter cannot run without the decryption filter, because it's the unencrypted data that gets compressed. encrypted objects are replicated in their encrypted form, so we preserve their initial compression attribute instead of trying to de/recompress

Fixes: https://tracker.ceph.com/issues/57905

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows

[cbodley]

in teuthology, multisite tests use compression type 'random', but local multisite testing doesn't support compression so i had to hack it in:

diff --git a/src/test/rgw/test_multi.py b/src/test/rgw/test_multi.py
index 57d27343efc..9c19537f667 100644
--- a/src/test/rgw/test_multi.py
+++ b/src/test/rgw/test_multi.py
@@ -347,6 +347,11 @@ def init(parse_args):
                 if len(endpoints):
                     arg += ['--endpoints', ','.join(endpoints)]
                 zone.create(cluster, arg)
+                # enable compression
+                cluster.admin(zone.zone_args() + [
+                    'zone', 'placement', 'modify',
+                    '--placement-id', 'default-placement',
+                    '--compression', 'zlib'])
             else:
                 zone.get(cluster)
             zonegroup.zones.append(zone)

with that change, i'm able to reproduce the ZlibCompressor: Decompression error: decompress return -3 errors on replication that cause the test_encrypted_object_sync test case to fail with a checkpoint timeout

with the fix applied, replication succeeds and the test case passes

[cbodley]

jenkins test api

[cbodley]

jenkins test api

[cbodley]

'passed' qa in rerun https://pulpito.ceph.com/cbodley-2023-06-30_13:12:49-rgw-wip-cbodley-testing-distro-default-smithi/

there were still some dead jobs due to Error reimaging machines: [Errno 104] Connection reset by peer, but none of those prevented the multisite tests from running. the two multisite jobs showed the normal test failures, but passed test_encrypted_object_sync which had been failing/crashing in https://tracker.ceph.com/issues/57905