message: optimazation for message priority strategy

[mslovy]

want peering and reply as fast as it can

Signed-off-by: Ning Yao yaoning@unitedstack.com

[athanatos]

Not this one, ReplicatedBackend bases the priority on the Push priority.

[athanatos]

@mslovy

PushReply has it's priority already set in ReplicatedBackend based on the Push priority -- that's better than setting it to HIGH unconditionally. Remove the PushReply change.

The two OpReply messages already have HIGH set in ReplicatedBackend. It's confusing to set it in both places. If you want to switch it to be set in the constructor, please separate that change into its own commit and remove the set_priority calls from ReplicatedBackend.

Setting the peering message priority in the constructor does make some sense, but don't do it in the same commit as the OpReply messages.

[mslovy]

@athanatos , updated， please review again. I think it is not reasonable to set the priority for MOSDSubOpReply since it is also used by others (like scrubbing). So I revert all modification related to Reply Ops, it would be fine and sometime more instructive to guide us setting different priority in different cases.
Optimize priority setting for kick_recovery_op

[athanatos]

@mslovy That second commit should be a separate PR. Currently, there are no tests at all that have different clients at different priorities, I would be surprised if it worked. That is work that needs to be done. The first step would be to add into the ceph-qa-suite rados/thrash suite a workload with two ceph_test_rados instances using different client priorities. Once we have that to test against, we can start merging patches to fix that behavior.

[mslovy]

@athanatos , reset the second commit and I will propose a new PR later as mentioned

[athanatos]

Ok, this makes sense, adding the needs-qa label.