gradio causing slow generation

[pseudotensor]

E.g. Put contents of this page into chat: https://www.emergentmind.com/ and hit enter

Once done, put this in: https://raw.githubusercontent.com/h2oai/h2o-llmstudio/main/LICENSE and hit enter

See slow generation and heavy CPU usage:

and very slow generation:

1 core usage, and alot of attempts by gradio to handle as image, tokenize it, parse it, etc.:

[pseudotensor]

E.g. can get stuck/slow in:

        elif isinstance(chat_message, str):
            children = self.md.parseInline(chat_message)[0].children
            if children and any("code" in child.tag for child in children):
                return self.md.render(chat_message)
            else:
                chat_message = chat_message.replace("\n", "<br>")
                return self.md.renderInline(chat_message)
        else:
            raise ValueError(f"Invalid message for Chatbot component: {chat_message}")

[pseudotensor]

This hack fixes it:

        elif isinstance(chat_message, str):
            children = self.md.parseInline(chat_message)[0].children
            if children and any("code" in child.tag for child in children):
                return chat_message # self.md.render(chat_message)
            else:
                chat_message = chat_message.replace("\n", "<br>")
                return chat_message  #self.md.renderInline(chat_message)
        else:
            raise ValueError(f"Invalid message for Chatbot component: {chat_message}")

But I don't see option to disable markdown processing gradio, and a bit odd that is so slow. They must have loops in loops or something.

[pseudotensor]

Still shows formatted output in gradio output despite no md processing:

[pseudotensor]

fixed with newer gradio