Add the option to use tuned model in shark_runner #79

yzhang93 · 2022-06-01T21:23:33Z

Add a flag to load pre-tuned model config file in shark_runner, and nothing is changed in inference api.
The example is made for minilm model with tensorflow frontend and GPU. To run the example:
python -m shark.examples.shark_inference.minilm_tf --device="gpu" --model_config_path=shark/examples/shark_inference/minilm_tf_gpu_config.json
The config file (example/shark-inference/.json) does not work for Torch frontend directly. The torch-mlir uses linalg.batch_matmul (instead of matmul) with first dimension as 1. So one needs to generate a new config file, and append the first dimension of tile_sizes as 1. Further test is needed for model annotation with Torch frontend.

powderluv

maybe we should name the model_config.json the same as the model name tf_minilm_config.json or something

yzhang93 · 2022-06-01T21:57:26Z

maybe we should name the model_config.json the same as the model name tf_minilm_config.json or something

Good point. Done.

raikonenfnu · 2022-06-01T22:30:59Z

shark/examples/shark_inference/minilm_tf_gpu_config.json

@@ -0,0 +1 @@
+{"options": [{"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [1, 32, 128], "work_group_sizes": [32, 1, 1], "pipeline": "GPU"}]}


Can we add this as a gsutil cp call?

In this case, where the file should be stored? I think people from outside may also want to run the command.

Let this example be checked in. We can wget other configs in the future.

raikonenfnu · 2022-06-01T22:34:59Z

shark/iree_utils.py

@@ -158,7 +161,8 @@ def get_iree_module(module, device, input_type, args, func_name):
 def get_iree_compiled_module(module,
                             device: str,
                             frontend: str = "torch",
-                             func_name: str = "forward"):
+                             func_name: str = "forward",
+                             use_tuned_model: str = None):


NIT: Can we rename use_tuned_model to something like tuned_model_path or model_config_path, since it's more representative of what the variable store. Ideally would have the same name in shark parser for easy readability.

Sure, it's done.

powderluv · 2022-06-01T23:10:55Z

Do you also see the benchmark module report 1.66ms on CUDA A100 ?

yzhang93 · 2022-06-01T23:39:56Z

Do you also see the benchmark module report 1.66ms on CUDA A100 ?

Let me build the latest perf branch and check the performance. I've checked the generated annotated mlir file and looks good.

powderluv · 2022-06-01T23:48:30Z

you don't need to build the latest branch. the new venv should have it

yzhang93 · 2022-06-02T21:34:05Z

Do you also see the benchmark module report 1.66ms on CUDA A100 ?

I got 1.75ms on my A100 vm. Just solved the merge conflict, and this PR should be ready to merge.

yzhang93 added 2 commits June 1, 2022 17:54

Add the option to use tuned model in shark_runner

52ad0fb

Merge branch 'main' into main

6ae909a

powderluv requested a review from raikonenfnu June 1, 2022 21:41

powderluv reviewed Jun 1, 2022

View reviewed changes

Modify the config file name

b4d5c29

powderluv self-requested a review June 1, 2022 22:21

powderluv approved these changes Jun 1, 2022

View reviewed changes

raikonenfnu reviewed Jun 1, 2022

View reviewed changes

Minor change to the variable name

ca95f58

yzhang93 added 2 commits June 2, 2022 20:45

Update the config file to include pipeline depth

901b142

Merge remote-tracking branch 'upstream/main' into main

ae09157

powderluv merged commit 16c50ca into nod-ai:main Jun 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the option to use tuned model in shark_runner #79

Add the option to use tuned model in shark_runner #79

yzhang93 commented Jun 1, 2022 •

edited

powderluv left a comment

yzhang93 commented Jun 1, 2022

raikonenfnu Jun 1, 2022

yzhang93 Jun 1, 2022

powderluv Jun 1, 2022

raikonenfnu Jun 1, 2022

yzhang93 Jun 1, 2022

powderluv commented Jun 1, 2022

yzhang93 commented Jun 1, 2022 •

edited

powderluv commented Jun 1, 2022

yzhang93 commented Jun 2, 2022

		@@ -0,0 +1 @@
		{"options": [{"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 32, 16], "work_group_sizes": [64, 2, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [32, 64, 32], "work_group_sizes": [128, 1, 1], "pipeline": "GPU_TENSORCORE"}, {"work_group_tile_sizes": [1, 64, 64, 32], "work_group_sizes": [128, 2, 1], "pipeline": "GPU_TENSORCORE", "split_k": 8}, {"work_group_tile_sizes": [1, 32, 128], "work_group_sizes": [32, 1, 1], "pipeline": "GPU"}]}

Add the option to use tuned model in shark_runner #79

Add the option to use tuned model in shark_runner #79

Conversation

yzhang93 commented Jun 1, 2022 • edited

powderluv left a comment

Choose a reason for hiding this comment

yzhang93 commented Jun 1, 2022

raikonenfnu Jun 1, 2022

Choose a reason for hiding this comment

yzhang93 Jun 1, 2022

Choose a reason for hiding this comment

powderluv Jun 1, 2022

Choose a reason for hiding this comment

raikonenfnu Jun 1, 2022

Choose a reason for hiding this comment

yzhang93 Jun 1, 2022

Choose a reason for hiding this comment

powderluv commented Jun 1, 2022

yzhang93 commented Jun 1, 2022 • edited

powderluv commented Jun 1, 2022

yzhang93 commented Jun 2, 2022

yzhang93 commented Jun 1, 2022 •

edited

yzhang93 commented Jun 1, 2022 •

edited