Is there an example of Chainlit working with Microsoft Guidance

[skprasadu]

Microsoft Guidance seem to be a powerful prompt engineering tool. Is there a working example of how chainlit works with this.

I came up with an example, but not sure if this is a best practice.

@cl.on_chat_start
def start_chat():
    guidance.llm = guidance.llms.OpenAI(...)

@cl.on_message
async def main(message: str):
    program = guidance('''
        {{#system~}}
        You are a helpful assistant
        {{~/system}}

        {{~#geneach 'conversation' stop=False}}
        {{#user~}}
        {{set 'this.user_text' (await 'user_text')  hidden=False}}
        {{~/user}}

        {{#assistant~}}
        {{gen 'this.ai_text' temperature=0 max_tokens=300}}
        {{~/assistant}}
        {{~/geneach}}''')

    program = program(user_text =message)
    
    print(program['conversation'])

    await cl.Message(content=str(program)).send()

This works great. But I get below prompt in my frontend. Not sure if there is a way to format this.

Chatbot

09:37:33 PM

<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
hi there<|im_end|>

    <|im_start|>assistant

Any help will be appreciated.

[willydouhard]

I was not familiar with this library. I played a bit with it and here is how it can integrate with Chainlit at first glance:

import chainlit as cl
import guidance
from guidance._program import Log

guidance.llm = guidance.llms.OpenAI(model="gpt-3.5-turbo")


class ChainlitLog(Log):
    def append(self, entry):
        super().append(entry)
        print(entry)
        is_end = entry["type"] == "end"
        is_assistant = entry["name"] == "assistant"
        if is_end and is_assistant:
            cl.run_sync(cl.Message(content=entry["new_prefix"]).send())


@cl.on_message
async def main(message: str):
    program = guidance(
        """
        {{#system~}}
        You are a helpful assistant
        {{~/system}}

        {{~#geneach 'conversation' stop=False}}
        {{#user~}}
        {{set 'this.user_text' (await 'user_text')  hidden=False}}
        {{~/user}}

        {{#assistant~}}
        {{gen 'this.ai_text' temperature=0 max_tokens=300}}
        {{~/assistant}}
        {{~/geneach}}""",
    )

    program(user_text=message, log=ChainlitLog())

This is very minimal and we have to explore what exactly is available in the entry dict. Also some string cleaning will be necessary.