From b5152ec4fbbcead5990b8821117813131d974832 Mon Sep 17 00:00:00 2001 From: Ashish Trivedi Date: Wed, 18 Jun 2025 11:47:59 -0700 Subject: [PATCH] @FIR-757: Update SDK to 0.1.4 and update release to 0.0.3 for tsi-ggml The test results with ./run_llama_cli.sh with 5 tokens is as follows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit +++ root@agilex7_dk_si_agf014ea:/usr/bin/tsi/v0.1.1.tsv31_06_06_2025/bin# ./run_llama_cli.sh my cat's name is Max. He' llama_perf_sampler_print: sampling time = 111.70 ms / 11 runs ( 10.15 ms per token, 98.47 tokens per second)llama_perf_context_print: load time = 132926.48 ms llama_perf_context_print: prompt eval time = 109957.33 ms / 6 tokens (18326.22 ms per token, 0.05 tokens per second) llama_perf_context_print: eval time = 195682.91 ms / 4 runs (48920.73 ms per token, 0.02 tokens per second) llama_perf_context_print: total time = 328764.01 ms / 10 tokens GGML Tsavorite Profiling Results: ------------------------------------------------------------------------------------------------------------------------ Calls Total(ms) T/call Self(ms) Function ------------------------------------------------------------------------------------------------------------------------ 33160 100086.000 3.018 47907.157 [32%] RuntimeHostShim::awaitCommandListCompletion 18920 29912.952 1.581 29912.952 └─ [10%] [ txe_silu ] 14080 22010.102 1.563 22010.102 └─ [ 7%] [ txe_mult ] 160 253.071 1.582 253.071 └─ [ 0%] [ txe_add ] 33160 1.178 0.000 1.178 └─ [ 0%] TXE 0 Idle 1 114.000 114.000 18.000 [ 0%] GGML Tsavorite 1 96.000 96.000 96.000 └─ [ 0%] RuntimeHostShim::initialize 1 52.000 52.000 52.000 [ 0%] RuntimeHostShim::finalize 33160 26.000 0.001 26.000 [ 0%] RuntimeHostShim::loadBlob 33160 23.000 0.001 23.000 [ 0%] RuntimeHostShim::finalizeCommandList 33160 5.000 0.000 5.000 [ 0%] RuntimeHostShim::addCommandToList 33161 3.000 0.000 3.000 [ 0%] RuntimeHostShim::allocate 33160 3.000 0.000 3.000 [ 0%] RuntimeHostShim::createCommandList 113720 0.000 0.000 0.000 [ 0%] RuntimeHostShim::getShmemManager 33160 0.000 0.000 0.000 [ 0%] RuntimeHostShim::launchBlob 33160 0.000 0.000 0.000 [ 0%] RuntimeHostShim::unloadBlob 33160 0.000 0.000 0.000 [ 0%] RuntimeHostShim::deallocate ======================================================================================================================== 412163 308849.000 0.749308849.000 [100%] TOTAL ======================================================================================================================== root@agilex7_dk_si_agf014ea:/usr/bin/tsi/v0.1.1.tsv31_06_06_2025/bin# +++ --- tsi-pkg-build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tsi-pkg-build.sh b/tsi-pkg-build.sh index d2f09ffd2e001..64c577235b911 100755 --- a/tsi-pkg-build.sh +++ b/tsi-pkg-build.sh @@ -6,7 +6,7 @@ echo 'updating submodule' git submodule update --recursive --init cd ggml-tsi-kernel/ module load tsi4 gcc/13.3.0 -export MLIR_SDK_VERSION=/proj/rel/sw/sdk-r.0.1.3 +export MLIR_SDK_VERSION=/proj/rel/sw/sdk-r.0.1.4 echo 'creating python virtual env' /proj/local/Python-3.10.12/bin/python3 -m venv blob-creation source blob-creation/bin/activate @@ -48,7 +48,7 @@ cmake --build build-fpga --config Release echo 'creating tar bundle for fpga' -TSI_GGML_VERSION=0.0.2 +TSI_GGML_VERSION=0.0.3 TSI_GGML_BUNDLE_INSTALL_DIR=tsi-ggml GGML_TSI_INSTALL_DIR=ggml-tsi-kernel TSI_GGML_RELEASE_DIR=/proj/rel/sw/ggml/