Streaming to file leads to "duplicates"

[kchro3]

Hi all,

First of all, amazing work on this project. I'm really excited about what you're doing.

There's something I noticed about the way the ./stream works with the file parameter. I believe that for the stdout, it's only printing to a newline every n_new_line, and it would otherwise use a carriage return to overwrite the last line. This works nicely for stdout, but with the file output parameter, it's printed as a new line for every iteration.

I think this is buggy because the resulting text file has a bunch of duplicate lines most of the time, and it's difficult to parse out later. I'm guessing that this was written this way because you can't write carriage returns to a file and have it clear the last line like you can with stdout, but I think it would be a good compromise to only write to file on every n_new_line.

What do you think?

[ggerganov]

I mainly did it this way to be able to have real-time text in Vim:

whisper.vim.mp4

But, we can add a command line option to achieve what you suggest.

[zachrattner]

@ggerganov @kchro3 I am also interested in parsing the streaming output and getting only unique utterances in the output. Is there a recommended approach to take here?

[svyatoclav]

@ggerganov @kchro3 @zachrattner same here. I need some good way to properly parse the ./bin/stream output because -f output.txt prints a lot of unfinished and random lines in the file and I am not a programmer so idk how to correctly do this without a lot of headache so I can send it to local LLM (KoboldCPP API)...