Added SmartGPT workflow (issue #4463)

[UmerHA]

Added SmartGPT workflow by providing SmartLLM wrapper around LLMs

Edit:
As @hwchase17 suggested, this should be a chain, not an LLM. I have adapted the PR.

It is used like this:

from langchain.prompts import PromptTemplate
from langchain.chains import SmartLLMChain
from langchain.chat_models import ChatOpenAI

hard_question = "I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?"
hard_question_prompt = PromptTemplate.from_template(hard_question)

llm = ChatOpenAI(model_name="gpt-4")
prompt = PromptTemplate.from_template(hard_question)
chain = SmartLLMChain(llm=llm, prompt=prompt, verbose=True)

chain.run({})

Original text:
Added SmartLLM wrapper around LLMs to allow for SmartGPT workflow (as in https://youtu.be/wVzuvf9D9BU). SmartLLM can be used wherever LLM can be used. E.g:

smart_llm = SmartLLM(llm=OpenAI())
smart_llm("What would be a good company name for a company that makes colorful socks?")

or

smart_llm = SmartLLM(llm=OpenAI())
prompt = PromptTemplate(
    input_variables=["product"],
    template="What is a good name for a company that makes {product}?",
)
chain = LLMChain(llm=smart_llm, prompt=prompt)
chain.run("colorful socks")

SmartGPT consists of 3 steps:

1. Ideate - generate n possible solutions ("ideas") to user prompt
2. Critique - find flaws in every idea & select best one
3. Resolve - improve upon best idea & return it

Fixes #4463

Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

• @hwchase17
• @agola11

Twitter: @UmerHAdil | Discord: RicChilligerDude#7589

[leo-gan]

Looks great!
Is it possible to add some real results to the example, please? The example should show that it works and produces very helpful results.

[hwchase17]

i think this is probably more of chain than an LLM (in the same way the constitutional AI chain exists

[UmerHA]

Agreed - will change

[UmerHA]

Requires #5030 to be merged. Will re-request review, when it's merged.

[leo-gan]

Sorry, I didn't get it.
To measure 6 liter you have to fill in the 6 liter jug. That's all.
The answer definitely didn't provide a good answer.
Probably, the question can be like the puzzle for the "Die hard" move:
Given a 5-gallon jug and a 3-gallon jug. Please, fill one with exactly 4 gallons of water.

[UmerHA]

Hey, I disagree :) This is the example shown in the original SmartGPT video (https://youtu.be/wVzuvf9D9BU?t=384). The point is none of the original ideas work at all (even with GPT4!), only with the SmartGPT workflow do we get a correct answer.

But I agree this is confusing without context. I added more explanation.

[UmerHA]

Hey @hwchase17 any more comments?

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Aug 11, 2023 10:43pm

[batmanscode]

Nice work! 😃

[UmerHA]

Hi @hinthornw! Also pinging you here, as you're the maintainer for Agents :) Any feedback?

[batmanscode]

@UmerHA do you think this could be used for a pandas/xorbits agent? To break down complex questions before answering

[UmerHA]

@UmerHA do you think this could be used for a pandas/xorbits agent? To break down complex questions before answering

Sure, why not. This is a general approach and should improve any task (at the cost of more token usage).

[batmanscode]

@UmerHA do you think this could be used for a pandas/xorbits agent? To break down complex questions before answering

Sure, why not. This is a general approach and should improve any task (at the cost of more token usage).

Interesting thanks. Must look into using this as an agent with tools!

[baskaryan]

apologies for the delay in reviewing, this is very cool! thank you @UmerHA!!