Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
mds/server: skip unwanted dn in handle_client_readdir #12870
We can skip unwanted dn which < (offset_key, snap) via map.lower_bound, rather than
Previously we iterate and skip dn which < (offset_key, dn->last), as dn->last >= snap
Signed-off-by: Xiaoxi Chen email@example.com
The initiative for this commit is, we hit MDS CPU bottleneck (100% on one core as it is single thread) in our cephFS production enviroment.
Troubleshooting showing one applications are calling readdir often, what is worth, the dir has 130K files. Profiling showing on average each readdir call take ~20ms to finish in this scale(#files) and waste a significant time(and CPU!) on skipping unwanted dentries.
Take this request as example: 10 ms was spent on skipping the dentries and 7 ms was spent on encoding the wanted 1024 dentries. This patch addressed the 10ms and attempt to minimize it.
2017-01-09 19:49:03.023878 7ff8d74ce700 10 mds.0.server snapid head
[Iterating and skipping all dentry < offset]
2017-01-09 19:49:03.033836 7ff8d74ce700 10 mds.0.cache.ino(100000edea3) encode_inodestat issuing pAsLsXsFscr seq 1867
[Finished encoding, reply to client,total size ~300KB + 17.752ms]