Analyze performance of voice transcribe and text summary

[flowac]

In voice/voice.py the transcribe to summary pipeline is slow. Takes about 10 seconds to fully process 60 seconds of audio input.
Find out if it's possible to use a different transcriber or summarizer to speed things up.

[flowac]

On 4070 Ti:

data/terry.mp3: 182 words
Summary: 64 words, 35.16483516483517 percent of total
As it turns out, my compiler started as a simple stack machine. And then over the years I converted over to SISC as I may add it. I combined together, I put optimizers to combine together some of my little risk instructions. And I could have had really a terrible code that was just risk. And it probably would have run just the same.
{'label': 'amusement', 'score': 0.4817044138908386}
{'label': 'disappointment', 'score': 0.3517039716243744}
{'label': 'realization', 'score': 0.07218757271766663}
{'label': 'annoyance', 'score': 0.06411561369895935}
Total: 4.395450115203857
Transcribe: 2.0238449573516846
Tokenize: 0.0016772747039794922
Summarize: 2.3374810218811035
Decode: 0.0006868839263916016
4395.492792129517 ms
Result: density = exceptional, impact = high

data/micro-machine.wav: 162 words
Summary: 31 words, 19.135802469135804 percent of total
Each one comes with its own special edition micro machine vehicle. Perfect pocket portable to take any place. Each one has dramatic details for a picture of a precision paint job.
{'label': 'approval', 'score': 0.45931702852249146}
{'label': 'admiration', 'score': 0.20619387924671173}
Total: 2.665823221206665
Transcribe: 1.3742787837982178
Tokenize: 0.0016727447509765625
Summarize: 1.278937816619873
Decode: 0.00046563148498535156
2665.8644676208496 ms
Result: density = good, impact = low

[flowac]

Patch for timing:

diff --git a/voice/voice.py b/voice/voice.py
index 8aba9b6..e5f8026 100644
--- a/voice/voice.py
+++ b/voice/voice.py
@@ -1,4 +1,4 @@
-import whisper
+import whisper, time
 from transformers import pipeline, BartTokenizer, BartForConditionalGeneration

 class voice:
@@ -46,11 +46,16 @@ class voice:

     # do not call directly
     def _run(self, fname):
+        t0 = time.time()
         result = self.voice2text.transcribe(fname)
+        t1 = time.time()
+
         word_count_total = len(result["text"].split())
         print(f'\n{fname}: {word_count_total} words')

         inputs = self.tokenizer(result["text"], return_tensors="pt", truncation=True, max_length=1024)
+        t2 = time.time()
+
         summary_ids = self.summarizer.generate(
             inputs.input_ids,
             num_beams=4,              # Beam search for higher quality
@@ -59,9 +64,12 @@ class voice:
             early_stopping=True,      # Stop early if possible
             no_repeat_ngram_size=3    # Avoid repeating 3-grams
         )
+        t3 = time.time()

         # Decode the generated summary
         summary = self.tokenizer.decode(summary_ids[0], skip_special_tokens=True)
+        t4 = time.time()
+
         word_count_summary = len(summary.split())
         word_percent = word_count_summary * 100 / word_count_total
         print(f'Summary: {word_count_summary} words, {word_percent} percent of total')
@@ -82,6 +90,8 @@ class voice:
                     pos_count = pos_count + 1
                 else:
                     neg_count = neg_count + 1
+        t5 = time.time()
+        print(f'Total: {t5-t0}\nTranscribe: {t1-t0}\nTokenize: {t2-t1}\nSummarize: {t3-t2}\nDecode: {t4-t3}')

         return self.score(word_count_total, word_percent, pos_count, neg_count)