Enable worker core pinning in CPU nightly benchmark

[min-jean-cho]

This PR enables worker core pinning by default for better cpu performance.

Below is a nightly cpu benchmark performance comparison:

default: https://github.com/pytorch/serve/actions/runs/4238841923
OMP_NUM_THREADS=1: https://github.com/pytorch/serve/actions/runs/4255376669
core pinning, --use_logical_cores: https://github.com/pytorch/serve/actions/runs/4320222197

[msaroufim]

Just retriggered CI - @min-jean-cho I noticed CI failures last week and a bunch of upstream changes to torch, do you need any help resolving this PR?

[min-jean-cho]

Thanks @msaroufim, let me convert this PR to WIP. The current status of this PR is to address the concerns brought up by @lxning when deploying this in production. I'll update this when ready, thanks!

[codecov]

Codecov Report

Merging #2166 (86a6898) into master (a460afb) will not change coverage.
The diff coverage is n/a.

❗ Current head 86a6898 differs from pull request most recent head 26d5c39. Consider uploading reports for the commit 26d5c39 to get more accurate results

@@           Coverage Diff           @@
##           master    #2166   +/-   ##
=======================================
  Coverage   71.41%   71.41%           
=======================================
  Files          73       73           
  Lines        3348     3348           
  Branches       57       57           
=======================================
  Hits         2391     2391           
  Misses        954      954           
  Partials        3        3           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[lxning]

do we need add numactl into docker container for cpu?

[msaroufim]

Same question as the one Li asked, otherwise LGTM. Approving because the this isn't breaking the docker CI

In our current dev docker images we do indeed run the install dependencies script https://github.com/pytorch/serve/blob/master/docker/Dockerfile.dev#L70 but since we're moving to using the regular docker images moving forward should also add numactl there - cc @agunapal

[min-jean-cho]

We can add numactl in docker cpu as well. But I recall in docker, numactl required privileged mode -- taskset is available for in this case.

[agunapal]

@min-jean-cho Could you please tell me where numactl is being used here.
Trying to understand why docker CI is not complaining.

[min-jean-cho]

Could you please tell me where numactl is being used here

@agunapal numactl is used internally by the launcher here, here.

Trying to understand why docker CI is not complaining.

I think because if cpu_launcher_enable=true and numactl is not available, then enabling launcher just skips. When we upgrade to the next torch 2.x release, if cpu_launcher_enable=true and numactl is not available, then then launcher automatically tries taskset here instead of just skipping.