librbd: require exclusive lock for reads if pwl cache is enabled

[majianpeng]

TestLibRBD.TestFUA descript the following workload:
a)write/read the same image w/ pwl-cache
write_image = open(image_name);
read_image = open(image_name);
b)i/o workload is:
write(write_image)
write need EXLock and require EXLOCK

read(read_image)
in ExclusiveLock::init(), firstly read need EXLOCK
so will require EXLOCK. write_image release EXLOCK(will
flush data to osd and remove cache). read_image init pwl-cache
and read-io firstly enter pwl-cache and missed and then read
from osd.

write(write_image)
write need EXLOCK and require EXLOCK. This make read_image remove
empty cache. write_image init cache pool and write data to cache.

read(read_image)
In send_set_require_lock(), it set write need EXLOCK.
So read don't require EXLOCK and dirtyly read from osd.

Because second-read don't need EXLOCK and make write_image don't
release EXLOCK(flush dirty data to osd and shutdown pwl-cache).
This make second-read don't read the latest data.

So we should make read also need EXLOCK when enable pwl-cache.

Fixes: https://tracker.ceph.com/issues/51438

Tested-by: Feng Hualong hualong.feng@intel.com
Signed-off-by: Jianpeng Ma jianpeng.ma@intel.com

Checklist

• References tracker ticket
• Updates documentation if necessary
• Includes tests for new functionality or reproducer for bug

---

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox

[majianpeng]

jenkins test make check

[majianpeng]

@idryomov . although make check faild, i wanted you can riew this. I'm not sure for this and it's a policy of exclusive-lock for cache.
Thanks!

[idryomov]

Seems completely reasonable to me. Please remove the RFC tag and sort out the unittest.