Feature request: stdout callback

[Venthe]

Hello; I'm currently writing a web UI for the ChatGPT4All. I've faced an issue where the only way for me to get output is to consume it at end. I'd like to request a change which will allow to pass a stdout callback function - which might be done without breaking backwards compatibility.

It could be done similar to this: https://github.com/Venthe/chatgpt4all-webui/blob/main/server/prompt_parser.py

I've also added ID there but it is of co consequence, as it can be a part of the callback. Same thing with type, as I've ended up doing it indempotently.

Proposal for the callback signature:
'''python
def callback(responseCharacter)
'''

Where the default could be as it is now, sys.stdout.print

[bmschmidt]

I'd recommend using the [pyllama bindings]( https://github.com/nomic-ai/pyllamacpp) for this, it includes a token-level callback function, and we will be moving the nomic gpt4all wrapper to rely on those bindings soon from the current setup.

…

On Wed, Apr 5, 2023 at 6:53 AM Jacek Lipiec ***@***.***> wrote: Hello; I'm currently writing a web UI for the ChatGPT4All. I've faced an issue where the only way for me to get output is to consume it at end. I'd like to request a change which will allow to pass a stdout callback function - which might be done without breaking backwards compatibility. It could be done similar to this: https://github.com/Venthe/chatgpt4all-webui/blob/main/server/prompt_parser.py I've also added ID there but it is of co consequence, as it can be a part of the callback. Same thing with type, as I've ended up doing it indempotently. Proposal for the callback signature: '''python def callback(responseCharacter) ''' Where the default could be as it is now, sys.stdout.print — Reply to this email directly, view it on GitHub <#128>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIPFZUHLU4U6PLJRZ3GC3DW7VFKFANCNFSM6AAAAAAWT5BSAQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[therealjuanmartinez]

I came to request the same - glad to see there's already some emphasis towards this

[Venthe]

@bmschmidt if so, then I believe I can close the issue. Thanks!