LiteLLM+LangFuse Support: Enhancing how request_params are overridden

[theobjectivedad]

This enhancement performs a recursive merge of request_params set initially on the agent with parameters passed in during an execution step. Consider the example below that adds a session identifier at runtime to a specific step in the process:

@app.agent(
    name="Generator",
    request_params=RequestParams(
        maxTokens=DEFAULT_MAX_TOKENS,
        use_history=False,
        metadata={
            "metadata": {
                "trace_name": "Novelist",
                "tags": TAGS,
            },
        },
    )
async def main() -> None:
  async with app.run() as agent:
    # ... Do something here
   # Generate a unique session identifier for the next steps...
    session_id = str(uuid4()).replace("-", "")

    # Now pass that session identifier to the agent...
    result = agent.Optimizer.structured(
      prompt,
      SomePydanticObject,
      request_params=RequestParams(
      metadata={"metadata": {"session_id": session_id}}
    )

Prior to this update, the inner metadata would be set to {"session_id": session_id} and basically would overwrite the default values set above on the agent. With this update session_id would be merged and the metadata would get set to:

metadata = {
  "metadata": {
    "trace_name": "Novelist",
    "tags": TAGS,
    "session_id": session_id,
  }
}

The specific use case I ran into was that I am using LiteLLM and LangFuse. It is easy enough to set metadata and pass in the specifics needed for tracing when calling @app.agent. However, there are some situations - like adding a session identifier or other execution specific details - where it isn't feasible to set these globally when initializing the agent decorator, and doing a "deep merge" that overrides the agent defaults is needed.

Note that although this is a small change and likely the behavior most folks would expect, it does introduce a potentially breaking change for any clients that rely on the "shallow" merge.

[alpeshvas]

the title is so misleading here.

[theobjectivedad]

Hiya @alpeshvas, apologies ... I'll own the bad title. LangFuse integration is the exact use case I had in mind though.

Here is how I am structuring my agent to use LangFuse. With this PR, LiteLLM+LangFuse will work more-or-less OOTB with FastAgent:

# See : https://docs.litellm.ai/docs/observability/langfuse_integration#trace--generation-parameters
TRACE_SESSION_ID = str(uuid4()).replace("-", "")
PARENT_TRACE_ID = str(uuid4()).replace("-", "")
TRACE_TAGS: list[str] = []

@app.agent(  # type: ignore
    name="Searcher",
    servers=["powersearch"],
    request_params=RequestParams(
        maxTokens=DEFAULT_MAX_TOKENS,
        use_history=False,
        parallel_tool_calls=True,
        metadata={
            "metadata": {
                "generation_name": "Searcher",
                "trace_id": PARENT_TRACE_ID,
                "trace_name": app.name,
                "tags": TRACE_TAGS,
                "session_id": TRACE_SESSION_ID,
            },
        },
    ),
)
async def main() -> None:
    """Main function to run the agent."""

    async with app.run() as agent:
        # Agent stuff goes here...