v. 1.6.7. Parakeet Speed-Up

[Tintin0]

Hi, I would like to understand how do you sped up Parakeet on Apple Silicon M4/M5 so significantly. Amazing achievement btw.

[altic-dev]

Appreciate it man, just some tweaking of how the operations run on Neural engine and fixing some low hanging fruits :)

[Tintin0]

Yeah, Parakeet is amazing in this regard. I would like to see also an reasonable good LLM run on neural engine too. I'm running a live meeting transcriber which summarizes and recommends questions etc. The transcription via Parakeet on neural engine is amazing, LLMs however are so GPU heavy and therefore draw a lot of energy which is next to pure energy consumption a fan noise problem.