rgw/notification: remove non x-amz-meta-* attributes from bucket notifications

[jzhu116-bloomberg]

Do not populate x-amz-meta-* attributes to the metadata section of the notification events.

Fixes: https://tracker.ceph.com/issues/59592
Signed-off-by: Jane Zhu jzhu116@bloomberg.net

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows

[jzhu116-bloomberg]

Tests look good. An example:
{ "Records": [ { "eventVersion": "2.2", "eventSource": "ceph:s3", "awsRegion": "zg1", "eventTime": "2023-05-01T23:04:59.686582Z", "eventName": "ObjectCreated:Put", "userIdentity": { "principalId": "zone.user" }, "requestParameters": { "sourceIPAddress": "" }, "responseElements": { "x-amz-request-id": "ae3edd61-a8e2-47de-9ad5-f184a718c267.4158.12721181013944665849", "x-amz-id-2": "103e-zg1-1-zg1" }, "s3": { "s3SchemaVersion": "1.0", "configurationId": "notif1", "bucket": { "name": "fish", "ownerIdentity": { "principalId": "zone.user" }, "arn": "arn:aws:s3:zg1::fish", "id": "ae3edd61-a8e2-47de-9ad5-f184a718c267.4160.1" }, "object": { "key": "myfile", "size": 52428800, "eTag": "63e8040a86587b5d0f044eba23bd06d4", "versionId": "Kinu5q1XLiAC9DW9yuqJKTSlkkIXGyy", "sequencer": "9B455064E965CE2E", "metadata": [ { "key": "x-amz-meta-foo", "val": "bar" }, { "key": "x-amz-meta-ka", "val": "boom" } ], "tags": [ { "key": "Key1", "val": "Value1" } ] } }, "eventId": "1682982299.785278.63e8040a86587b5d0f044eba23bd06d4", "opaqueData": "" } ] }

[mattbenjamin]

the notifications feature has automated tests; could we add such a test there?

[jzhu116-bloomberg]

the notifications feature has automated tests; could we add such a test there?

will do

[yuvalif]

the condition for being a metadata attribute is in line 646:

boost::algorithm::starts_with(attr.first, RGW_ATTR_META_PREFIX)

also, not clear to me how these attributes get into the reservation object in the first place. since the above condition should make sure only metadata attributes are inserted to the map

[mattbenjamin]

if the map isn't general, then yes, it would be preferable to not add them ;)

[jzhu116-bloomberg]

made the changes to stop populating non-metadata attributes to the reservation object

[jzhu116-bloomberg]

the notifications feature has automated tests; could we add such a test there?

It looks we don't need a new test case for this. I just added metadata checking in verify_s3_records_by_elements.

[jzhu116-bloomberg]

@mattbenjamin, @yuvalif, it looks like the failure in ceph API tests has nothing to do with the changes. I don't have the privilege to re-run it in the jenkins pipeline. Can either of please re-trigger it?

[yuvalif]

jenkins test api

[yuvalif]

teuthology passing: http://pulpito.front.sepia.ceph.com/yuvalif-2023-05-09_04:48:52-rgw:notifications-wip-yuval-test-distro-default-smithi/