os/bluestore: trim cache every 50ms (instead of 200ms)

[liewegas]

In small cache size situations trimming needs to be more frequent. See
https://tracker.ceph.com/issues/22616

This isn't a complete solution: in very low memory situations an even lower
value would be needed, or perhaps bluestore_default_buffered_read=false.

Signed-off-by: Sage Weil sage@redhat.com

[rzarzynski]

I would love to see cache trimming triggered on event, not only on elapsed time since last swept.

[ifed01]

I'm a bit nervous about the performance impact caused by this change...

[liewegas]

On one hand we should expect to trim 1/4 as many items 4x as often, so no net change. And it means the trimming spikes will be shorter, so the tail latency should improve.

On the other hand, there may be some fix overhead of the trim process itself. (I'm not really sure what, though.) FWIW back in the beginning we were trimming on every request. I don't think we ever did a comparison with the sharded cache change, though.

[tchaikov]

http://pulpito.ceph.com/kchai-2018-02-28_11:49:25-rados-wip-kefu-testing-2018-02-28-1653-distro-basic-smithi/

[tchaikov]

@liewegas i am adding backport=luminous to the corresponding ticket. please let me know if i am wrong.

[liewegas]

thanks!

[xiexingguo]

FYI: actually our local test results reveal a measurable 8k-random-read/write performance regression with this patch applied..

Was:

Test result (bandwidth: MB/s, latency: ms):
         type    bandwidth            iops         latency
---------------------------------------------------------------------
     1M_write          730             730             215
      1M_read         1225            1225             130
        1M_rw          886             886             179
 8K_randwrite           58            7531              21
  8K_randread           73            9357              17
    8K_randrw          121           15713               9
     8K_write          103           13214              12
      8K_read          211           27101               5
        8K_rw          187           23956               5
---------------------------------------------------------------------

Now:

Test result (bandwidth: MB/s, latency: ms):
         type    bandwidth            iops         latency
---------------------------------------------------------------------
     1M_write          727             727             216
      1M_read         1230            1230             130
        1M_rw          885             885             179
 8K_randwrite           55            7093              22
  8K_randread           69            8946              17
    8K_randrw          115           14859              10
     8K_write           98           12587              12
      8K_read          212           27182               5
        8K_rw          187           24052               5
---------------------------------------------------------------------