Can filters outlet be used for streaming chunks?

[thiswillbeyourgithub]

Hi,

I made a pipeline filter that removed the <thinking> tags used by my llm to improve relevance. It's great and all but only gets triggered after the last chunk was received, it then updates the message and removes the thinking block.

Is there a way to filter the streaming chunks directly? That would allow me to use audio chat/call mode without it having to read its internal thoughts :) it would also feel more natural overall as the ui wouldn't have to get updated.

I took a look at the examples but none seem to adress this. And I couldn't make proper pipelines work and don't really want to because they are still WIP

[tjbck]

You'd want to have a custom pipe function here!

[thiswillbeyourgithub]

Thanks but then the docs seems misleading at this page : https://docs.openwebui.com/pipelines/filters

Finally, the filter performs post-processing on the outgoing LLM message (or "outlet") before it is sent to the user.

It's not really what's happening if it's sent to the user by chunk and then the message gets "updated".

I probably sound like i'm nitpicking but really i'm just confused and this line didn't help :)

Also when you say "pipe function" does that mean that I can code this pipe in the "function" section of the workspace or does it need to be coded in a pipeline server? Because it seems to me that the pipe functions are actually python classes and that the functions refer to the heponymous workspace tab. I'm really confused... I swear I read multiple times the pipeline docs and the few examples in the repo but it's not helping so far :)

[thiswillbeyourgithub]

I think I've figured it out , thanks!

[thiswillbeyourgithub]

For anyone wondering my pipe is there: open-webui/pipelines#198 (reply in thread)

I succeeded in using a pipe function (not a pipeline) that dynamically changes the API used. Used with litellm's virtual keys this makes me able to track the cost of my users. Also removes "``` thinking" blocks :)

[zebra-doom]

No you're not nitpicking - you can't because the OI's backend cannot accept AsyncGenerator being returned by the outlet.
So all fancy streaming should be done in pipe.

[Classic298]

OUTLET now being called - in dev.