Launcher core pinning #1401

min-jean-cho · 2022-01-13T17:53:53Z

Incorporates launchers' new feature --instance_idx to TorchServe which helps improve the performance of multi-worker inference via core pinning.
Updates README for intel_extension_for_pytorch. Updates include: 1) useful launcher's configurations; 2) comment on how instance_idx improves performance; 3) ipex performance boost data

Note that currently users have to install IPEX from source to use --instance_idx feature.

msaroufim · 2022-01-13T18:15:09Z

examples/intel_extension_for_pytorch/README.md

@@ -65,6 +66,17 @@ Some useful `cpu_launcher_args` to note are:

 Please refer to [here](https://github.com/intel/intel-extension-for-pytorch/blob/master/docs/tutorials/performance_tuning/launch_script.md) for a full list of tunable configuration of launcher. 

+Some notable launcher configurations are:


So we should mention that installation from source is required. If we don't have to wait too long it may make sense to wait for an official IPEX release before we merge this PR. Thoughts?

I agree on waiting until the next IPEX release prior to merging this PR.

Hi all, Intel® Extension for PyTorch* 1.11.0-cpu has been released few days ago. I think we can rezoom this PR. It would be good to merge this PR prior or soon after our upcoming blog. Thanks. cc. @msaroufim @lxning

msaroufim · 2022-01-13T18:16:31Z

examples/intel_extension_for_pytorch/README.md

@@ -65,6 +66,17 @@ Some useful `cpu_launcher_args` to note are:

 Please refer to [here](https://github.com/intel/intel-extension-for-pytorch/blob/master/docs/tutorials/performance_tuning/launch_script.md) for a full list of tunable configuration of launcher. 

+Some notable launcher configurations are:
+1. `--ninstances`: Number of instances for multi-instance inference/training.  


could use a few words to explain the difference between a worker and instance

I thought they are interchangeable. @ashokei is there a difference?

"worker" is a torchserve terminology, we should clarify explicitly how worker and launcher instance are related.

I have removed this section to not confuse the readers. It's taken internally by launcher. Please see my comment below as well.

msaroufim · 2022-01-13T18:17:04Z

examples/intel_extension_for_pytorch/README.md

+When running multi-worker inference with Torchserve, launcher's `--ninstances`, `--instance_idx`, and `--ncore_per_instance` boost the performance of Torchserve.
+Launcher equally divides the number of cores by the number of workers such that each worker is pinned to assigned cores. Doing so avoids core overlap between workers which generally improves performance. 
+For example, assume running four workers (4 `ninstances`) on 16 cores (i.e., 4 `ncore_per_instance`). Launcher will bind worker 0 (`instance_idx` 0) to cores 0-3, worker 1 (`instance_idx` 1) to cores 4-7, worker 2 (`instance_idx` 2) to cores 8-11, and worker 3 (`instance_idx` 3) to cores 12-15.
+Note that core pinning is taken care internally by launcher such that users don't have to manually set `ninstances`, `instance_idx`, nor `ncore_per_instance` in `config.properties`.


This is an important point. If this is the case is it still worth mentioning the 3 parameters in the doc? Perhaps best to link to IPEX docs and just say something along the lines that CPU launcher will take care of core pinning

Done. I have removed the 3 parameters that are taken care internally to not confuse the readers.

msaroufim · 2022-01-13T18:17:26Z

examples/intel_extension_for_pytorch/README.md

+Launcher equally divides the number of cores by the number of workers such that each worker is pinned to assigned cores. Doing so avoids core overlap between workers which generally improves performance. 
+For example, assume running four workers (4 `ninstances`) on 16 cores (i.e., 4 `ncore_per_instance`). Launcher will bind worker 0 (`instance_idx` 0) to cores 0-3, worker 1 (`instance_idx` 1) to cores 4-7, worker 2 (`instance_idx` 2) to cores 8-11, and worker 3 (`instance_idx` 3) to cores 12-15.
+Note that core pinning is taken care internally by launcher such that users don't have to manually set `ninstances`, `instance_idx`, nor `ncore_per_instance` in `config.properties`.
+


Would be helpful to test out if this works in a docker image as well

Did we still want to test out in a docker image? Please let me know. Thanks. cc @msaroufim @lxning

Yes because need to make sure that the current docker image has access to the same environment variables

launcher internally restarts the workers to re-distribute cores that were pinned to killed workers to the remaining.
@min-jean-cho I was wondering if this means serving will be interrupted for the time this re-distribution is taking place?

msaroufim · 2022-01-13T18:18:40Z

examples/intel_extension_for_pytorch/README.md

+```
+ipex_enable=true
+cpu_launcher_enable=true
+cpu_launcher_args=--socket_id 0 --ninstance 1 --enable_jemalloc


I thought we didn't need to set this in config.properties or is this for reproducibility reasons?

It was for reproducibility, but indeed --ninstance 1 can be omitted. However, --socket_id 0 --enable_jemalloc should be set as they are not set by default.

Have removed --ninstance 1.

msaroufim · 2022-01-13T18:19:46Z

examples/intel_extension_for_pytorch/README.md

+
+![pdt_perf](https://github.com/min-jean-cho/frameworks.ai.pytorch.ipex-cpu-1/assets/93151422/a158ba6c-a151-4115-befb-39acb7545936)
+
+Above shows performance improvement of Torchserve with IPEX and launcher on ResNet50 and BERT-base-uncased. Torchserve official [apache-bench benchmark](https://github.com/pytorch/serve/tree/master/benchmarks#benchmarking-with-apache-bench) on Amazon EC2 m6i.24xlarge was used to collect the results. Add the following lines in ```config.properties``` to reproduce the results. Notice that launcher is configured such that a single instance uses all physical cores on a single socket to avoid cross socket communication and core overlap. 


I was wondering if we could add more details as to why m6i and c6i should be the recommended choice, what is it about those machines thatt make them run Pytorch faster? I'd even mention this at the very top since benefits are bigger than any software changes

I had used m6i for benchmarking purpose of this data, not for performance comparison of m6i vs c6i. I'm not sure about the recommended choice from performance perspective.

@ashokei could you please comment on this as well?

right, we were not necessarily benchmarking c6 vs m6 , rather we were using "24xlarge" which has most cores. We could add general comment about using multi-core instances, how ipex optimizations automatically scale and leverage full instance resources.

Thanks for the clarification. Makes sense.

msaroufim · 2022-01-13T18:22:11Z

examples/intel_extension_for_pytorch/README.md

+
+For example, run the following command to reproduce latency performance of ResNet50 with data type of IPEX int8 and batch size of 1. 
+```
+python benchmark-ab.py --url 'file:///model_store/rn50_ipex_int8.mar' --concurrency 1


would need to add more detail on how to get this mar file? Or put a link to something like an S3 bucket or gdrive

I guess I didn't make it clear that the rn50_ipex_int8.mar is created from steps shown here: https://github.com/intel/intel-extension-for-pytorch/blob/master/docs/tutorials/performance_tuning/torchserve.md#2-creating-a-model-archive

msaroufim · 2022-01-13T18:36:09Z

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java

+
+            if (isRestartWorkers) {
+                logger.warn("restarting {} thread(s)", restartNumWorkers);
+                addThreads(threads, model, restartNumWorkers, future);


Could you explain a bit more about the overall logic of this PR - it seems to do (my very rough understanding)

Check if CPU launcher available and if so 0 out number of workers

Then add threads back to model set to the old min worker

Finally when a new process is launched make sure it includes the important CLI arguments like n-instance and instance-idx

Workers are restarted only when cpu_launcher is enabled (does not affect Torchserve when cpu_launcher_is not enabled). Restarting workers is useful/needed when user scales the number of workers up/down during execution ( https://pytorch.org/serve/management_api.html#scale-workers ) By restarting workers, launcher 1) re-allocates the cores that were pinned to killed workers in case of scale down; 2) avoids core overlap in case of scale up. It kills all existing workers and restarts scaled up/down number of workers. By doing so, launcher is configured properly when user scales the number of workers during execution. I will add comments to the file to explain the logic.

msaroufim · 2022-01-13T18:37:54Z

examples/intel_extension_for_pytorch/README.md

+
+## Performance Boost with IPEX and Launcher
+
+![pdt_perf](https://github.com/min-jean-cho/frameworks.ai.pytorch.ipex-cpu-1/assets/93151422/a158ba6c-a151-4115-befb-39acb7545936)


getting a 404 here

Should be fixed now. Thanks.

min-jean-cho · 2022-01-13T19:55:03Z

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java

@@ -85,13 +85,24 @@ public int getNumRunningWorkers(ModelVersionName modelVersionName) {
        return numWorking;
    }

+    public boolean isLauncherRestartWorkers(int currentWorkers) {
+        boolean isRestart;
+        if (configManager.isCPULauncherEnabled() && currentWorkers > 0) {


checks cpu_launcher is enabled and currentWorkers is greater than 0 (i.e., only restarts workers when scaling workers up/down; if currentWorkers==0, then workers are being initialized)

min-jean-cho · 2022-01-13T20:33:43Z

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java

+                        currentWorkers,
+                        minWorker);
+                maxWorker = 0;
+                minWorker = 0;


By setting minWorker=0 when currentWorkers > 0, this will never enter the if (currentWorkers < minWorker) condition in line 134. And by setting maxWorker=0, the for loop in line 137 will proceed to remove all currentWorkers.

min-jean-cho · 2022-01-13T20:38:04Z

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java

    public CompletableFuture<Integer> modelChanged(
            Model model, boolean isStartup, boolean isCleanUp) {
        synchronized (model.getModelVersionName()) {
            boolean isSnapshotSaved = false;
            CompletableFuture<Integer> future = new CompletableFuture<>();
            int minWorker = model.getMinWorkers();
            int maxWorker = model.getMaxWorkers();
+            int restartNumWorkers = minWorker;


Sets restartNumWorkers to updated minWorker after scale up/down.

min-jean-cho · 2022-01-13T20:45:02Z

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java

+
+            if (isRestartWorkers) {
+                logger.warn("restarting {} thread(s)", restartNumWorkers);
+                addThreads(threads, model, restartNumWorkers, future);


Finally after removing all currentNumWorkers of workers, add back (i.e., restart) restartNumWorkers. Feel free to have a try with apache benchmark. For example, setting initial workers to 1 and scaling up to 5 via curl -v -X PUT "http://localhost:8081/models/benchmark?min_worker=5" or setting initial workers to 5 and scaling down to 1 curl -v -X PUT "http://localhost:8081/models/benchmark?min_worker=1"

…eaders (2) updated socket_id to node_id (3) added launcher core pinning section

msaroufim · 2022-04-01T18:59:02Z

examples/intel_extension_for_pytorch/README.md

+Launcher equally divides the number of cores by the number of workers such that each worker is pinned to assigned cores. Doing so avoids core overlap between workers which generally improves performance. 
+For example, assume running four workers (4 `ninstances`) on 16 cores (i.e., 4 `ncore_per_instance`). Launcher will bind worker 0 (`instance_idx` 0) to cores 0-3, worker 1 (`instance_idx` 1) to cores 4-7, worker 2 (`instance_idx` 2) to cores 8-11, and worker 3 (`instance_idx` 3) to cores 12-15.
+Note that core pinning is taken care internally by launcher such that users don't have to manually set `ninstances`, `instance_idx`, nor `ncore_per_instance` in `config.properties`.
+


Yes because need to make sure that the current docker image has access to the same environment variables

min-jean-cho · 2022-04-06T21:18:08Z

Hi all, I have tested core pinning with docker image - please see the attached image. Couple of notes:

(1) IPEX 1.11 requires PyTorch 1.11. For now, I have tested by modifying requirements/torch_linux.txt as here. Seems like upgrading to PyTorch 1.11 is in plan #1434 .

(2) added numactl and ipex installation in Dockerfile.dev as here.
If you prefer not to make direct changes to Dockerfile.dev, (2.1) we may add a parameter such as --build_with_ipex to build_image.sh, or (2.2) user can install ipex inside their docker container themselves. The benefit of option 2.1 is that users don't have to worry themselves.

(3) user need to start the docker container with --privileged mode for numactl core binding to work
For example, docker run --privileged --rm -it -p 8080:8080 -p 8081:8081 -p 8082:8082 -p 7070:7070 -p 7071:7071 launcher_core_pinning

Let me know of your thoughts. Thanks. cc @msaroufim @lxning @ashokei

HamidShojanazeri · 2022-04-11T15:39:28Z

examples/intel_extension_for_pytorch/README.md

+Launcher equally divides the number of cores by the number of workers such that each worker is pinned to assigned cores. Doing so avoids core overlap between workers which generally improves performance. 
+For example, assume running four workers (4 `ninstances`) on 16 cores (i.e., 4 `ncore_per_instance`). Launcher will bind worker 0 (`instance_idx` 0) to cores 0-3, worker 1 (`instance_idx` 1) to cores 4-7, worker 2 (`instance_idx` 2) to cores 8-11, and worker 3 (`instance_idx` 3) to cores 12-15.
+Note that core pinning is taken care internally by launcher such that users don't have to manually set `ninstances`, `instance_idx`, nor `ncore_per_instance` in `config.properties`.
+


launcher internally restarts the workers to re-distribute cores that were pinned to killed workers to the remaining.
@min-jean-cho I was wondering if this means serving will be interrupted for the time this re-distribution is taking place?

HamidShojanazeri · 2022-04-11T15:41:52Z

examples/intel_extension_for_pytorch/README.md

+python benchmark-ab.py --url {modelUrl} --input {inputPath} --concurrency 1 
+```
+
+For example, run the following command to reproduce latency performance of ResNet50 with data type of IPEX int8 and batch size of 1. 


it will be good to mention/ add a link to how int8 quantization was applied to these models.

Hi @HamidShojanazeri , that's a good question - serving is indeed paused/interrupted for few seconds while redistributing. But there's no code crash or anything. Let me know if this suffices or if you would like further experiments. Thanks

thanks @min-jean-cho, I think it might be good to clarify it in the doc.
cc: @lxning

Hi @HamidShojanazeri , I have added a gif and footnote to demonstrate that serving is interrupted for few seconds while re distributing cores. Please have a look at the updated README https://github.com/min-jean-cho/serve/tree/launcher_core_pinning/examples/intel_extension_for_pytorch#scaling-workers . Thanks

min-jean-cho · 2022-06-29T23:26:21Z

Hi @msaroufim @HamidShojanazeri , please let me know if there's any blocking on this PR

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java

Co-authored-by: Aaqib <maaquib@gmail.com>

msaroufim · 2022-07-13T17:04:08Z

Can confirm test gets skipped on GPU

(launcher) ubuntu@ip-172-31-16-198:~/serve/test/pytest$ pytest test_example_intel_extension_for_pytorch.py 
=============================================================================================================== test session starts ===============================================================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/ubuntu/serve
plugins: cov-3.0.0, mock-3.8.2
collected 4 items                                                                                                                                                                                                                                 

test_example_intel_extension_for_pytorch.py ssss                                                                                                                                                                                            [100%]

=============================================================================================================== 4 skipped in 2.05s ================================================================================================================
(launcher) ubuntu@ip-172-31-16-198:~/serve/test/pytest$ nvidia-smi
Wed Jul 13 17:03:47 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:00:1E.0 Off |                    0 |
| N/A   35C    P0    23W / 300W |      0MiB / 16384MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

- port https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-cpu/pull/740 to `run_cpu` - use-case by pytorch/serve#2166 where `numactl` is unavailable (e.g., requires `privileged` mode) This PR automatically tries taskset if numactl core binding doesn't work. Reference: `taskset` is added to adapt to launcher use-cases such as in docker where `numactl` requires to be ran in `privileged` mode, where the `privileged` mode "wont work for deployments like sagemaker for example" as raised by TorchServe. Please see [torchserve ipex docker discussion](pytorch/serve#1401 (comment)) for reference. To address such use-cases, `taskset` can be used in place of `numactl` to set core affinity. Note that, unlike `numactl`, `taskset` does not provide memory binding to local memories; however, memory binding may not be needed in these use-cases that typically do not span multi sockets. Hence we can automatically try taskset if numactl doesn't work. Pull Request resolved: #96011 Approved by: https://github.com/jgong5, https://github.com/malfet

- port https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-cpu/pull/740 to `run_cpu` - use-case by pytorch/serve#2166 where `numactl` is unavailable (e.g., requires `privileged` mode) This PR automatically tries taskset if numactl core binding doesn't work. Reference: `taskset` is added to adapt to launcher use-cases such as in docker where `numactl` requires to be ran in `privileged` mode, where the `privileged` mode "wont work for deployments like sagemaker for example" as raised by TorchServe. Please see [torchserve ipex docker discussion](pytorch/serve#1401 (comment)) for reference. To address such use-cases, `taskset` can be used in place of `numactl` to set core affinity. Note that, unlike `numactl`, `taskset` does not provide memory binding to local memories; however, memory binding may not be needed in these use-cases that typically do not span multi sockets. Hence we can automatically try taskset if numactl doesn't work. Pull Request resolved: pytorch#96011 Approved by: https://github.com/jgong5, https://github.com/malfet

- port https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-cpu/pull/740 to `run_cpu` - use-case by pytorch/serve#2166 where `numactl` is unavailable (e.g., requires `privileged` mode) This PR automatically tries taskset if numactl core binding doesn't work. Reference: `taskset` is added to adapt to launcher use-cases such as in docker where `numactl` requires to be ran in `privileged` mode, where the `privileged` mode "wont work for deployments like sagemaker for example" as raised by TorchServe. Please see [torchserve ipex docker discussion](pytorch/serve#1401 (comment)) for reference. To address such use-cases, `taskset` can be used in place of `numactl` to set core affinity. Note that, unlike `numactl`, `taskset` does not provide memory binding to local memories; however, memory binding may not be needed in these use-cases that typically do not span multi sockets. Hence we can automatically try taskset if numactl doesn't work. Pull Request resolved: pytorch/pytorch#96011 Approved by: https://github.com/jgong5, https://github.com/malfet

- port https://github.com/intel-innersource/frameworks.ai.pytorch.ipex-cpu/pull/740 to `run_cpu` - use-case by pytorch/serve#2166 where `numactl` is unavailable (e.g., requires `privileged` mode) This PR automatically tries taskset if numactl core binding doesn't work. Reference: `taskset` is added to adapt to launcher use-cases such as in docker where `numactl` requires to be ran in `privileged` mode, where the `privileged` mode "wont work for deployments like sagemaker for example" as raised by TorchServe. Please see [torchserve ipex docker discussion](pytorch/serve#1401 (comment)) for reference. To address such use-cases, `taskset` can be used in place of `numactl` to set core affinity. Note that, unlike `numactl`, `taskset` does not provide memory binding to local memories; however, memory binding may not be needed in these use-cases that typically do not span multi sockets. Hence we can automatically try taskset if numactl doesn't work. Pull Request resolved: pytorch#96011 Approved by: https://github.com/jgong5, https://github.com/malfet

min-jean-cho and others added 2 commits January 12, 2022 15:49

added core pinning feature

52bdf66

added comment on instance_idx improving perf

9ffaaf5

msaroufim requested review from msaroufim and lxning January 13, 2022 18:12

msaroufim requested changes Jan 13, 2022

View reviewed changes

min-jean-cho commented Jan 13, 2022

View reviewed changes

msaroufim mentioned this pull request Feb 23, 2022

[RFC]: TorchServe Model Analyzer #1457

Closed

8 tasks

min-jean-cho and others added 9 commits March 18, 2022 09:47

Merge branch 'pytorch:master' into launcher_core_pinning

d95e7aa

(1) removed ninstances, instance_idx that's internal to not confuse r…

3417c90

…eaders (2) updated socket_id to node_id (3) added launcher core pinning section

Merge branch 'master' into launcher_core_pinning

d8b5cce

commented restartWorkers logic

bcfcfa8

minor grammar change

c78f8ce

added scaling workers

9eb94b6

fixed java formatting

4620e08

updated REAMDE

ac01674

updated REAME

9d0c035

msaroufim requested changes Apr 1, 2022

View reviewed changes

Merge branch 'pytorch:master' into launcher_core_pinning

3901593

HamidShojanazeri reviewed Apr 11, 2022

View reviewed changes

msaroufim self-requested a review April 11, 2022 17:58

msaroufim approved these changes Apr 11, 2022

View reviewed changes

msaroufim self-requested a review April 11, 2022 17:59

min-jean-cho added 4 commits April 11, 2022 11:26

added int8 links

3d86c01

fix to Intel® Extension for PyTorch*

7ebdc91

added link to int8

d34bd5c

added steps to creating bert_ipex_int8.mar

290dac3

msaroufim requested a review from HamidShojanazeri May 23, 2022 18:44

min-jean-cho added 4 commits May 25, 2022 14:52

Merge branch 'pytorch:master' into launcher_core_pinning

7c0af9d

include (1) core pinning (2) core pinning with worker scaling gifs

23385a7

add worker scaling footnote

c848f16

Merge branch 'master' into launcher_core_pinning

24ae718

maaquib approved these changes Jun 30, 2022

View reviewed changes

frontend/server/src/main/java/org/pytorch/serve/wlm/WorkLoadManager.java Outdated Show resolved Hide resolved

min-jean-cho and others added 2 commits June 30, 2022 09:53

simplify isLauncherRestartWorkers

72c1ca9

Co-authored-by: Aaqib <maaquib@gmail.com>

lint with pre-commit

ef850d3

HamidShojanazeri approved these changes Jun 30, 2022

View reviewed changes

min-jean-cho added 11 commits June 30, 2022 14:21

dummy commit for ci

a123334

undo dummy commit for ci

d06cc8c

fix minor shift

d7f82a0

check launcher available

57d34cd

add pytest checkif windows

574653c

added ipex to requirements/deverloper.txt

cb489ee

check if ipex installed

1a7fd9a

fixed mistake

86a160c

updated to ipex launcher available

50eaaa0

change naming

4e484b5

change dummy cmd to ls

2827603

msaroufim merged commit 16d5e2b into pytorch:master Jul 13, 2022

msaroufim mentioned this pull request Jul 20, 2022

Don't install IPEX on Mac to unblock CI #1745

Merged

3 tasks

min-jean-cho mentioned this pull request Mar 6, 2023

enable taskset core pinning in addition to numactl pytorch/pytorch#96011

Closed

		@@ -65,6 +66,17 @@ Some useful `cpu_launcher_args` to note are:

		Please refer to [here](https://github.com/intel/intel-extension-for-pytorch/blob/master/docs/tutorials/performance_tuning/launch_script.md) for a full list of tunable configuration of launcher.

		Some notable launcher configurations are:


		![pdt_perf](https://github.com/min-jean-cho/frameworks.ai.pytorch.ipex-cpu-1/assets/93151422/a158ba6c-a151-4115-befb-39acb7545936)

		Above shows performance improvement of Torchserve with IPEX and launcher on ResNet50 and BERT-base-uncased. Torchserve official [apache-bench benchmark](https://github.com/pytorch/serve/tree/master/benchmarks#benchmarking-with-apache-bench) on Amazon EC2 m6i.24xlarge was used to collect the results. Add the following lines in ```config.properties``` to reproduce the results. Notice that launcher is configured such that a single instance uses all physical cores on a single socket to avoid cross socket communication and core overlap.


		## Performance Boost with IPEX and Launcher

		![pdt_perf](https://github.com/min-jean-cho/frameworks.ai.pytorch.ipex-cpu-1/assets/93151422/a158ba6c-a151-4115-befb-39acb7545936)

Launcher core pinning #1401

Launcher core pinning #1401

Conversation

min-jean-cho commented Jan 13, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho Mar 18, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho Jan 13, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho Jan 13, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho Jan 13, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho commented Apr 6, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho Apr 11, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

min-jean-cho commented Jun 29, 2022

msaroufim commented Jul 13, 2022

min-jean-cho Mar 18, 2022 •

edited

min-jean-cho Jan 13, 2022 •

edited

min-jean-cho Jan 13, 2022 •

edited

min-jean-cho Jan 13, 2022 •

edited

min-jean-cho Apr 11, 2022 •

edited