Best practice for MCPTools with stateful multi-pod MCP gateway (Lunar MCPx) — per-run session routing

[user1500177]

Context & Architecture

We're building a production AI agent system using agno with the following stack:

• agno 2.5.16 (Python)
• FastAPI application with AgentOS + AG-UI interface
• MCPTools (streamable-http transport) connecting to a Lunar MCPx stateful gateway
• Gateway runs on Kubernetes with 2+ pods behind a load balancer
• Deployed on Azure with Argo CD

Our Setup (Simplified)

from agno.os import AgentOS
from agno.os.config import AgentOSConfig, ChatConfig
from agno.os.interfaces.agui import AGUI
from agno.agent import Agent
from agno.team import Team
from agno.tools.mcp import MCPTools, StreamableHTTPClientParams
from fastapi import FastAPI
from contextlib import asynccontextmanager


# --- MCP Tools Factory ---
def get_mcp_gateway_tools(app_config, consumer_tag=None, config_service=None):
    """Creates MCPTools instance for gateway connection."""
    headers = {
        "User-Id": app_config.app_user_id_gateway or "Agent",
    }
    if consumer_tag:
        headers["x-lunar-consumer-tag"] = consumer_tag

    server_params = StreamableHTTPClientParams(
        url=app_config.mcp_gateway_url,
        headers=headers,
        timeout=90.0,
    )
    return MCPTools(
        server_params=server_params,
        transport="streamable-http",
        timeout_seconds=90,
        refresh_connection=True,
    )


# --- Agent Creation ---
read_agent = Agent(
    name="read-agent",
    model=my_model,
    tools=[get_mcp_gateway_tools(app_config, consumer_tag="ctagent")],
)

write_agent = Agent(
    name="write-agent",
    model=my_model,
    tools=[get_mcp_gateway_tools(app_config, consumer_tag="ctagent")],
)


# --- Team ---
team = Team(
    name="vitebot",
    members=[read_agent, write_agent],
    model=my_model,
)


# --- FastAPI Lifespan ---
@asynccontextmanager
async def lifespan(app: FastAPI):
    # Connect all MCP tools ONCE at startup
    for agent in [read_agent, write_agent]:
        for tool in agent.tools:
            if hasattr(tool, "connect"):
                await tool.connect()  # Establishes persistent session

    # Mount AgentOS
    agent_os = AgentOS(
        id="vitebot-wms-os",
        name="ViteBot WMS",
        teams=[team],
        interfaces=[AGUI(team=team)],
    )
    app.mount("/v1/api", agent_os.get_app())

    yield

    # Shutdown: close connections
    for agent in [read_agent, write_agent]:
        for tool in agent.tools:
            if hasattr(tool, "close"):
                await tool.close()

app = FastAPI(lifespan=lifespan)

---

The Problem: Stateful MCP Gateway with Multiple Pods

Our MCP gateway (Lunar MCPx) is stateful — each session is bound to the pod that created it. When we have 2+ gateway pods:

1. tools.connect() at startup → creates a session on Pod A
2. All subsequent tool calls use that session
3. If the load balancer routes a request to Pod B → session doesn't exist there

Error we see on the agno app side:

ERROR    Failed to get tools for <MCPTools name=MCPTools functions=[]>
ERROR    Failed to initialize MCP toolkit
WARNING  Failed to build tools for <MCPTools name=MCPTools functions=[]>: Session terminated

Error on the MCP gateway side:

WARN: Session not found  sessionId="dec6b239-c20c-4eb3-adc0-d72cc08f6387"
      consumerTag="ctagent"

---

What We've Tried (DevOps-Side Routing Strategies)

Approach	Result
Source IP routing	Failed in Azure disaster recovery (private machines use masked IPs that change on failover)
Static User-Id header	Works but ALL requests go to the same pod forever — poor utilisation of 2-pod deployment

---

Questions

---

Is there a better way we can make sure that the both of the pods of the mcp server can be used ; this beacem a issue in stateful mcp server gateway like lunar

Isnt the Mcp-session-id passed as a header for the remaing BY AGNO if not shouldnt it be or IS it a mistake on my part ? - FEEL free to correct me- can we use that for the MAPPING TO THE CORRECT POD is that POSSIBLE or is this scenario arising due to wrong imeplemntation from my side ?
(https://zhimin-wen.medium.com/scaling-http-streamable-mcp-servers-on-kubernetes-handling-sticky-sessions-24212857c8ca)

Environment

• agno version: 2.5.16
• MCP gateway: Lunar MCPx (stateful, multi-pod on K8s)
• Transport: streamable-http

Related (Could dynamic headrs be used for this ? )

• PR fix: apply header_provider headers during MCP initialization #7507 (Dynamic headers for MCP)
• Docs: https://docs.agno.com/basics/tools/mcp/dynamic-headers

---

Thank you for any guidance! Happy to provide more details about our architecture.

[armorer-labs]

For a stateful multi-pod MCP gateway, I would treat “per-run routing” as a first-class part of the agent run context, not just an HTTP concern.

The pattern I would aim for:

• generate a stable run_id at the agent/workflow boundary
• propagate it to MCP gateway headers
• use it for sticky routing or gateway-side session lookup
• include tool_call_id for each invocation
• persist enough gateway/session metadata to debug later

If the gateway owns state, the agent framework should not have to guess which pod has it. Either sticky routing must be deterministic from the run/session key, or the gateway needs an external session store. Otherwise retries and resumed runs become very hard to reason about.

[user1500177]

Hey , @armorer-labs , @pandego , @kausmeows i am a starter in this Agent framework world

1)is the MCP server conneciton supposed to be made once on the pplication startup and then throughout the connection are supposed to be maintined - is that the best preforming one ?? (if thats the case how would run_id be the best - COULD you help me understand it with some clarity )

2)Have you heard about the refresh_connection parameter from AGNO - avaiblable in there MCPTools class , if i enable it is the list of tools gettting refreshed each time i make a tool call , if yes that would be a overhead right ? (#7810)

3)Also , if i wish to do this (Related to the multiPOD gateway)- what is CONSIDERED the best - at startup itslef , should be MAKING the connection to any one of the PODs right in that case i would need some header to distinguish and route it to the POD ,right and then throught all the tool calls made with that tool instace should go via it right ; In the case of run_id if i use (For each run especially for each USER PROMPT that literally translates to my CURRENT UNDERSTADING (CORRECT ME IF WRONG ) - then run or session key should be used for the routing right - IF POINT (1) is true HOW TO DO THIS is where all of MY DOUBT arise from - in agno i already found that at the startup iteslef the connection should be made in a PRODUCTION environement to my current understading(https://docs.agno.com/tools/mcp/overview)

WOULD BE HELPFUL if you/anyone could help me to understadn these perspectives //