Add language model base and work out interactions with planning manager

[collijk]

This PR adds the base abstraction for the language model and adjusts some of the interface of the Planning model to create their communication contract.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)			May 5, 2023 3:32pm

[collijk]

Some thoughts on how this will work:

# This may happen "client" application side before we create the Agent, I'm thinking

planner = Planner(...)
language_model = LanguageModel(...)

initial_user_input = ... # Application logic for this
objective_prompt = planner.construct_objective_prompt_from_user_input(initial_user_input)
model_response = language_model.determine_agent_objective(objective_prompt)

agent_objective = model_response.content  # This should be parsed into a standard format already

# This may happen on the "server" side (across the message broker interface)
# ...Update API budget, etc. ...



agent = Agent(
    agent_objective, 
    planner,
    language_model, 
    ...,
)
agent.run()

class Agent:

    def __init__(self, agent_objective, planner, language_model, ...):
        self.objective = agent_objective
        self.planner = planner
        self.language_model = language_model
        ...

    def run():
        while True:
            # Logic to construct the context (checking budget, looking for memories, getting user input)
            planning_prompt_context = PlanningPromptContext(...)
            planning_prompt = self.planner.construct_planning_prompt_from_context(planning_prompt_context)
            action_plan_response = self.language_model.plan_next_axtion(planning_prompt)

            action_plan = action_plan_response.content
            
            # ...Update API budget, check for user confirmation, update memory, etc.
            
            # Logic to construct the context ...
            self_feedback_prompt_context = SelfFeedbackPromptContext(...)
            self_feedback_prompt = self.planner.get_self_feedback_prompt(self_feedback_prompt_context)
            self_feedback_response = self.language_model.get_self_feedback(self_feedback_prompt)
            
            self_feedback = self_feedback_response.content

            # ...Update API budget, resolve action plan with self feedback, update memory, do next action, etc.

[Boostrix]

People interested in learning more about ongoing talks about the planning manager, refer to: #3593

[DarylRodrigo]

Just wondering, why are we importing the entire configuration here?

[collijk]

I'm starting permissive because we're going to have to port the existing logic into this system and we're going to find edge cases in that port that we can't isolate without significant work. We don't even have the most basic form of encapsulation yet. We can tighten contracts later once we have a little structure to lean on, and our priority in this work is getting that structure in quickly.

[DarylRodrigo]

Where do we add new models?

[collijk]

That's a good question and one I'll show in concrete implementation. Essentially the core system will define a couple of models. The plugin system may register new ones (eventually). Part of the language model definition will include concrete instances of LanguageModelInfo and EmbeddingModelInfo objects.

[Niraj-Kamdar]

Left some minor questions regarding interface

[Niraj-Kamdar]

Why are we doing this? Is there any recursive import dependencies?

[collijk]

There is an import dependency structure, it is currently unclear and in flux and will include cyclic dependencies, I'm proactively making sure the other developers contributing systems don't hit those cyclic dependencies an can work independently. We will resolve imports in the more standard way when all the work is joined.

[Niraj-Kamdar]

No need for ellipsis if docstrings. We can get rid of all ellipsis in favor of docstrings to make it more readable.

[collijk]

This is a nitpick and one I don't totally agree with. It also has no bearing on the PR content.