rgw: D3N: cache objects which oid contains slash

[mkogan1]

allow D3N caching of objects names for example:

sudo ./bin/rados ls -p default.rgw.buckets.data
...
ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse/cephs3/parkingcitations/parkingcitations_5.csv.2~wcGD_MF_W9hEBAV8x3l9uHKHKQC5_mn.88_1
ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse/cephs3/parkingcitations/parkingcitations_2.csv.2~eJyQZhfiayNv9Z-27_IfUb4mDwQnSmv.4_1
ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse/cephs3/parkingcitations/parkingcitations_3.csv.2~VY905gVNZ-nXxfBLgko84z1TUv_c74f.85_
                                                             ^      ^

after this PR:

ls -lh /mnt/nvme/rgw_datacache
...
-rw-r--r-- 1 root root 4.0M May 29 13:55 ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse%2Fcephs3%2Fparkingcitations%2Fparkingcitations_5.csv.2~wcGD_MF_W9hEBAV8x3l9uHKHKQC5_mn.99_1
-rw-r--r-- 1 root root 4.0M May 29 13:55 ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse%2Fcephs3%2Fparkingcitations%2Fparkingcitations_5.csv.2~wcGD_MF_W9hEBAV8x3l9uHKHKQC5_mn.99_2
-rw-r--r-- 1 root root 3.0M May 29 13:55 ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse%2Fcephs3%2Fparkingcitations%2Fparkingcitations_5.csv.2~wcGD_MF_W9hEBAV8x3l9uHKHKQC5_mn.99_3

and refactor to clean up the D3N storage location variables for uniformity.
(manual testing for the correctness of cache writing->reading->expiration were performed)

Fixes: https://tracker.ceph.com/issues/59739

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows

[mkogan1]

teuthology run results:
https://pulpito.ceph.com/mkogan-2023-05-30_12:17:25-rgw-wip-d3n-obj-w-slash_i01-distro-default-smithi/

[mkogan1]

@dang @mattbenjamin @samarahu @pritha-srivastava
FYI, just a heards up - object names with a 'slash' in the name can occur,
in this case they were created by Trino

ex:

sudo ./bin/rados ls -p default.rgw.buckets.data
...
ec43cdb3-59d1-4304-8511-9a3c180f048b.4231.3__shadow_warehouse/cephs3/parkingcitations/parkingcitations_3.csv.2~VY905gVNZ-nXxfBLgko84z1TUv_c74f.85_
                                                             ^      ^

[mkogan1]

jenkins test make check

[mkogan1]

jenkins test make check arm64

[github-actions]

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

[mkogan1]

rebased because of merge-conflicts

[mkogan1]

@cbodley is it OK to merge or does it need another teuthology run?

[mkogan1]

per discussion in the bug scrub, ok to merge