Skip to content

Conversation

@atrivedi-tsavoritesi
Copy link

Removed the echo output i.e. the first line when a command is pushed to serial console.

The output looks like this now for this URL

http://10.50.0.124:5003/llama-cli?model=tiny-llama&backend=tSavorite&tokens=10&prompt=My+cat%27s+name&repeat-penalty=1.5&batch-size=1024&top-k=50&top-p=0.9&last-n=5&context-length=12288&temp=0.0

My cat's name was Tim. He loved to play with his toy llama_perf_sampler_print: sampling time = 200.46 ms / 16 runs ( 12.53 ms per token, 79.82 tokens per second)llama_perf_context_print: load time = 1716.78 ms llama_perf_context_print: prompt eval time = 605.59 ms / 6 tokens ( 100.93 ms per token, 9.91 tokens per second) llama_perf_context_print: eval time = 1293.49 ms / 9 runs ( 143.72 ms per token, 6.96 tokens per second) llama_perf_context_print: total time = 3260.52 ms / 15 tokens GGML Tsavorite Profiling Results: ------------------------------------------------------------------------------------------------------------------------ Calls Total(ms) T/call Self(ms) Function ------------------------------------------------------------------------------------------------------------------------ 715 715.000 1.000 0.000 [17%] RuntimeHostShim::awaitCommandListCompletion 460 696.855 1.515 696.855 └─ [16%] [ txe_silu ] 245 365.281 1.491 365.281 └─ [ 8%] [ txe_mult ] 10 14.906 1.491 14.906 └─ [ 0%] [ txe_add ] 715 0.458 0.001 0.458 └─ [ 0%] TXE 0 Idle 1 34.000 34.000 34.000 [ 1%] RuntimeHostShim::finalize 1 15.000 15.000 1.000 [ 0%] GGML Tsavorite 1 14.000 14.000 14.000 └─ [ 0%] RuntimeHostShim::initialize 716 0.000 0.000 0.000 [ 0%] RuntimeHostShim::allocate 2400 0.000 0.000 0.000 [ 0%] RuntimeHostShim::getShmemManager 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::createCommandList 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::loadBlob 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::launchBlob 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::addCommandToList 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::finalizeCommandList 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::unloadBlob 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::deallocate ======================================================================================================================== 8838 4311.000 0.488 4311.000 [100%] TOTAL ========================================================================================================================

Copy link

@dineshReddy6381 dineshReddy6381 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

Copy link

@LewisLui777 LewisLui777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Job I like it.

@atrivedi-tsavoritesi atrivedi-tsavoritesi merged commit f5713b3 into master Jun 18, 2025
@atrivedi-tsavoritesi atrivedi-tsavoritesi deleted the FIR-756 branch June 18, 2025 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants