@FIR-756: Removed the echo of command in flask output #19
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Removed the echo output i.e. the first line when a command is pushed to serial console.
The output looks like this now for this URL
http://10.50.0.124:5003/llama-cli?model=tiny-llama&backend=tSavorite&tokens=10&prompt=My+cat%27s+name&repeat-penalty=1.5&batch-size=1024&top-k=50&top-p=0.9&last-n=5&context-length=12288&temp=0.0
My cat's name was Tim. He loved to play with his toy llama_perf_sampler_print: sampling time = 200.46 ms / 16 runs ( 12.53 ms per token, 79.82 tokens per second)llama_perf_context_print: load time = 1716.78 ms llama_perf_context_print: prompt eval time = 605.59 ms / 6 tokens ( 100.93 ms per token, 9.91 tokens per second) llama_perf_context_print: eval time = 1293.49 ms / 9 runs ( 143.72 ms per token, 6.96 tokens per second) llama_perf_context_print: total time = 3260.52 ms / 15 tokens GGML Tsavorite Profiling Results: ------------------------------------------------------------------------------------------------------------------------ Calls Total(ms) T/call Self(ms) Function ------------------------------------------------------------------------------------------------------------------------ 715 715.000 1.000 0.000 [17%] RuntimeHostShim::awaitCommandListCompletion 460 696.855 1.515 696.855 └─ [16%] [ txe_silu ] 245 365.281 1.491 365.281 └─ [ 8%] [ txe_mult ] 10 14.906 1.491 14.906 └─ [ 0%] [ txe_add ] 715 0.458 0.001 0.458 └─ [ 0%] TXE 0 Idle 1 34.000 34.000 34.000 [ 1%] RuntimeHostShim::finalize 1 15.000 15.000 1.000 [ 0%] GGML Tsavorite 1 14.000 14.000 14.000 └─ [ 0%] RuntimeHostShim::initialize 716 0.000 0.000 0.000 [ 0%] RuntimeHostShim::allocate 2400 0.000 0.000 0.000 [ 0%] RuntimeHostShim::getShmemManager 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::createCommandList 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::loadBlob 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::launchBlob 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::addCommandToList 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::finalizeCommandList 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::unloadBlob 715 0.000 0.000 0.000 [ 0%] RuntimeHostShim::deallocate ======================================================================================================================== 8838 4311.000 0.488 4311.000 [100%] TOTAL ========================================================================================================================