osd: Make dmclock's anticipation timeout be configurable

[TaewoongKim]

This adds a configuration option that can control anticipation timeout for dmclock

This helps more accurate QoS or priority based scheduling when dmclock is used with this.

By setting with an appropriate value, a client or an operation type could take their unused resource that could be forfeited by other aggressive clients or operation types.

Signed-off-by: Taewoong Kim taewoong.kim@sk.com

[myoungwon]

How is anticipation_timeout used ?

[TaewoongKim]

My bad. I missed something.
Thank you for pointing that. I fixed it.

[myoungwon]

@TaewoongKim This PR seems that only the anticipation timeout is set. Could you explain how this value is used ?

[TaewoongKim]

@myoungwon Yes, it just set dmclock's parameter
I requested PR of dmclock project about anticipation timeout.
It was merged a few weeks ago. (ceph/dmclock#43, ceph/dmclock#34)
Now, this PR is for enabling the dmclock anticipation timeout PR on Ceph.

Dmclock scheduler supports IOPS reservation.
However, an aggressive worker can take light woker's reserved shares.
Assume that worker A is a light worker whose IOPS reservation is 100 IOPS and worker B is an aggressive worker whose IOPS reservation is 1 IOPS.
Also assume that Woker A generates just 10 IOs in one second but every its IO does not come in exactly every 10 ms(arrived with a very small variation)
In this case, Worker A couldn't get serviced 10 IOPS, because worker B takes worker A's share.
If Worker A's IO is a little late(even 1ms) Worker B's IOs will be processed rather than Woker A's and Worker A's IO will be delayed.
(You can see an example in ceph/dmclock#34)

This is because dmclock reset the time tag of worker A's IO.
In a normal case, dmclock set time tag for IO based on previous IO's tag.
However, if an IO arrived more than (1/reserved IOPS) ms later since the previous IO that belongs to the same worker arrived,
the time tag of newly arrived IO is reset by the current time.

Setting anticipation timeout can prevent this situation.
Reset will be deferred by anticipation timeout and time tag will be set based on previous IO's tag.

[myoungwon]

@TaewoongKim need rebase

[TaewoongKim]

@myoungwon Rebased

[myoungwon]

@tchaikov @liewegas @ivancich Could you take a look? dmclock code related to this PR has already been merged.

[yuriw]

wip-yuri4-testing-2018-01-06-1732