-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Feat/fastapi modular server contribution #2952
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
FrigaZzz
wants to merge
9
commits into
google:main
from
FrigaZzz:feat/fastapi-modular-server-contribution
Closed
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
1593ca0
feat: add FastAPI modular server example with SSE streaming support
FrigaZzz 67e6bf5
Merge branch 'google:main' into feat/fastapi-modular-server-contribution
FrigaZzz 7beab8c
refactor: replace __import__("time") with time module import
FrigaZzz 0a3a0a4
refactor: improve code readability and maintainability
FrigaZzz 235bca7
refactor: improve type hints and code organization
FrigaZzz c393957
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz d4cc10c
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz a7ea7f3
style: reorganize imports for better readability and consistency (./aβ¦
FrigaZzz c5b3545
Merge branch 'main' into feat/fastapi-modular-server-contribution
FrigaZzz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # Application Configuration | ||
| DEBUG=true | ||
| PORT=8881 | ||
| HOST=localhost | ||
| LOG_LEVEL=INFO | ||
| LOGGING=true | ||
|
|
||
| # ADK Configuration | ||
| SERVE_WEB_INTERFACE=true | ||
| RELOAD_AGENTS=true | ||
|
|
||
| # Google Gemini Configuration | ||
| GOOGLE_API_KEY=YOUR_GOOGLE_API_KEY_HERE | ||
|
|
||
| # Model Configuration | ||
| MODEL_PROVIDER=google |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Logs | ||
| *.log | ||
|
|
||
| # Environment variables | ||
| .env | ||
|
|
||
| # Python bytecode | ||
| *.pyc | ||
| __pycache__/ | ||
|
|
||
| # Test artifacts | ||
| .pytest_cache/ | ||
|
|
||
| # Virtual environments | ||
| .venv/ | ||
| venv/ | ||
|
|
||
| # IDE configuration | ||
| .vscode/ | ||
| .idea/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,314 @@ | ||
| # π Google ADK FastAPI Modular Server | ||
|
|
||
| A **production-ready template** for extending Google's Agent Development Kit (ADK) with custom FastAPI endpoints, optimized SSE streaming, and modular architecture patterns. | ||
|
|
||
| ## π― Purpose & Value | ||
|
|
||
| The **FastAPI Modular Server** serves as a **template and reference implementation** for teams who want to: | ||
|
|
||
| - **Extend ADK's built-in server** without modifying core behavior | ||
| - **Accelerate production deployment** with battle-tested patterns | ||
| - **Add custom business logic** through modular router systems | ||
| - **Enable hot-reload capabilities** for faster development cycles | ||
|
|
||
| ## β¨ Key Features | ||
|
|
||
| ### π§ **Modular Router Architecture** | ||
| - Clean separation of concerns with dedicated router classes | ||
| - Easy to add new endpoints without touching core server code | ||
|
|
||
| ### β‘ **Optimized SSE Streaming** | ||
| - **3 optimization levels** for different use cases: | ||
| - `MINIMAL`: Essential content only (author + text) | ||
| - `BALANCED`: Core data with invocation tracking | ||
| - `FULL_COMPAT`: Complete ADK event compatibility | ||
| - Reduced payload sizes for improved performance | ||
| - Custom event filtering and mapping | ||
|
|
||
| ### π **Hot-Reload Development** | ||
| - Automatic agent reloading on file changes | ||
| - File system monitoring with `watchdog` | ||
| - Development-friendly with production stability | ||
|
|
||
|
|
||
| ## π Project Structure | ||
|
|
||
| ``` | ||
| fastapi_modular_server/ | ||
| βββ .env.example # Environment variables template | ||
| βββ README.md # Project documentation | ||
| βββ __init__.py # Package initialization | ||
| βββ app/ # Main application directory | ||
| β βββ __init__.py # App package initialization | ||
| β βββ agents/ # Agent definitions | ||
| β β βββ greetings_agent/ # Greetings agent module | ||
| β β βββ __init__.py # Agent package init | ||
| β β βββ greetings_agent.py # Greetings agent implementation | ||
| β βββ api/ # API layer | ||
| β β βββ __init__.py # API package init | ||
| β β βββ routers/ # API route definitions | ||
| β β β βββ __init__.py # Routers package init | ||
| β β β βββ agent_router.py # Agent-related API routes | ||
| β β βββ custom_adk_server.py # FastAPI server configuration | ||
| β βββ config/ # Configuration management | ||
| β β βββ settings.py # Application settings | ||
| β βββ core/ # Core application components | ||
| β β βββ __init__.py # Core package init | ||
| β β βββ dependencies.py # Dependency injection | ||
| β β βββ logging.py # Logging configuration | ||
| β β βββ mapping/ # Data mapping utilities | ||
| β β βββ __init__.py # Mapping package init | ||
| β β βββ sse_event_mapper.py # Server-Sent Events mapper | ||
| β βββ models/ # Data models | ||
| β βββ __init__.py # Models package init | ||
| β βββ streaming_request.py # Streaming data models | ||
| βββ main.py # Application entry point | ||
| ``` | ||
|
|
||
| ## π Quick Start | ||
|
|
||
| ### 1. **Configuration** | ||
| ```bash | ||
| # Copy environment template | ||
| cp .env.example .env | ||
|
|
||
| # Edit .env with your settings | ||
| vim .env | ||
|
|
||
| # Set the API KEY | ||
|
|
||
| ``` | ||
|
|
||
| ### 2. **Run the Server** | ||
| ```bash | ||
| # Development mode with hot-reload | ||
| python main.py | ||
|
|
||
| # Production mode | ||
| uvicorn main:app --host 0.0.0.0 --port 8881 | ||
| ``` | ||
|
|
||
|
|
||
| ## π§ Customization Guide | ||
|
|
||
| ### **Adding New Routers** | ||
|
|
||
| Create a new router following the established pattern: | ||
|
|
||
| ```python | ||
| # app/api/routers/my_custom_router.py | ||
| from fastapi import APIRouter, Depends | ||
| from app.core.dependencies import ADKServices, get_adk_services | ||
|
|
||
| class MyCustomRouter: | ||
| def __init__(self, web_server_instance): | ||
| self.web_server = web_server_instance | ||
| self.router = APIRouter(prefix="/custom", tags=["Custom"]) | ||
| self._setup_routes() | ||
|
|
||
| def _setup_routes(self): | ||
| @self.router.get("/endpoint") | ||
| async def my_endpoint( | ||
| ): | ||
| # Access any ADK service | ||
| sessions = await self.web_server.session_service.list_sessions() | ||
| return {"data": "custom response", "session_count": len(sessions)} | ||
|
|
||
| def get_router(self) -> APIRouter: | ||
| return self.router | ||
| ``` | ||
|
|
||
| Register it in the custom server: | ||
|
|
||
| ```python | ||
| # In app/api/server.py - CustomAdkWebServer class | ||
| def _initialize_routers(self): | ||
| try: | ||
| self.agent_router = AgentRouter(self) | ||
| self.my_custom_router = MyCustomRouter(self) # Add this | ||
| logger.info("All routers initialized successfully.") | ||
| except Exception as e: | ||
| logger.error(f"Failed to initialize routers: {e}", exc_info=True) | ||
|
|
||
| def _register_modular_routers(self, app: FastAPI): | ||
| # ... existing code ... | ||
|
|
||
| if self.my_custom_router: | ||
| app.include_router(self.my_custom_router.get_router()) | ||
| logger.info("Registered MyCustomRouter.") | ||
| ``` | ||
|
|
||
| ### **Overriding ADK Endpoints** | ||
|
|
||
| #### **Method 1: Route Removal (Current Approach)** | ||
|
|
||
| ```python | ||
| def _register_modular_routers(self, app: FastAPI): | ||
| # Remove specific ADK routes | ||
| routes_to_remove = [] | ||
| for route in app.routes: | ||
| if route.path in [ | ||
| "/run_sse", | ||
| "/apps/{app_name}/users/{user_id}/sessions" | ||
FrigaZzz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ] and hasattr(route, 'methods') and 'POST' in route.methods: | ||
| routes_to_remove.append(route) | ||
FrigaZzz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| # Remove the routes | ||
| for route in routes_to_remove: | ||
| app.routes.remove(route) | ||
| ``` | ||
|
|
||
| #### **Method 2: Middleware Interception** | ||
|
|
||
| For more complex overrides, use middleware: | ||
|
|
||
| ```python | ||
| from fastapi import Request, Response | ||
| from starlette.middleware.base import BaseHTTPMiddleware | ||
|
|
||
| class RouteOverrideMiddleware(BaseHTTPMiddleware): | ||
| async def dispatch(self, request: Request, call_next): | ||
| # Intercept specific routes | ||
| if request.url.path == "/run_sse" and request.method == "POST": | ||
| # Handle with custom logic | ||
| return await self.handle_custom_sse(request) | ||
|
|
||
| return await call_next(request) | ||
| ``` | ||
|
|
||
| ### **Accessing ADK Services and Runners** | ||
|
|
||
| #### **From Router Classes** | ||
| ```python | ||
| class AgentRouter: | ||
| def __init__(self, web_server_instance): | ||
| self.web_server = web_server_instance | ||
|
|
||
| async def my_endpoint(self, adk_services: ADKServices = Depends(get_adk_services)): | ||
| # Access services | ||
| agents = self.web_server.agent_loader.list_agents() | ||
| session = await self.web_server.session_service.list_sessions() | ||
|
|
||
| # Access runners through web server | ||
| runner = await self.web_server.get_runner_async("your_app_name") | ||
|
|
||
| # Access other web server properties | ||
| runners_cache = self.web_server.runners_to_clean | ||
| ``` | ||
|
|
||
|
|
||
|
|
||
| ### **Optimizing SSE Streaming** | ||
|
|
||
| #### **Custom Event Filtering** | ||
|
|
||
| Extend the SSE mapper for more sophisticated filtering: | ||
|
|
||
| ```python | ||
| # app/models/streaming_request.py | ||
| class OptimizationLevel(str, Enum): | ||
| """Enumeration for the available SSE optimization levels.""" | ||
|
|
||
| MINIMAL = "minimal" | ||
| BALANCED = "balanced" | ||
| FULL_COMPAT = "full_compat" | ||
| ULTRA_MINIMAL = "ultra_minimal" | ||
|
|
||
| # app/core/mapping/sse_mapper.py | ||
| class AdvancedSSEEventMapper(SSEEventMapper): | ||
| def map_event_to_sse_message(self, event: Event, optimization_level: OptimizationLevel) -> Optional[str]: | ||
| # Custom filtering logic | ||
| if self._should_skip_event(event): | ||
| return None | ||
|
|
||
| # Custom payload creation | ||
| payload = self._create_custom_payload(event, optimization_level) | ||
|
|
||
| # Custom serialization | ||
| return self._serialize_payload(payload) | ||
|
|
||
| def _should_skip_event(self, event: Event) -> bool: | ||
| # Skip system events, debug events, empty events, etc. | ||
| if event.author in ["system", "debug"]: | ||
| return True | ||
| if not event.content or not event.content.parts: | ||
| return True | ||
| return False | ||
|
|
||
| def _create_custom_payload(self, event: Event, level: OptimizationLevel) -> Dict: | ||
| if level == OptimizationLevel.ULTRA_MINIMAL: | ||
FrigaZzz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # Even more minimal than minimal | ||
| return {"t": self._extract_text_only(event)} | ||
|
|
||
| return super()._create_minimal_payload(event) | ||
| ``` | ||
|
|
||
| #### **Streaming Performance Optimizations** | ||
|
|
||
| 1. **Batch Events**: Combine multiple streaming events (single chunk) into a single SSE message to reduce overhead. | ||
| ```python | ||
| async def _generate_events_batched(self, req, sse_mapper, adk_services): | ||
| batch = [] | ||
| batch_size = 5 | ||
|
|
||
| async for event in self._get_events(): | ||
| batch.append(event) | ||
|
|
||
| if len(batch) >= batch_size: | ||
| # Process batch | ||
| combined_payload = self._combine_events(batch) | ||
| yield f"data: {json.dumps(combined_payload)}\n\n" | ||
| batch.clear() | ||
| ``` | ||
|
|
||
| 2. **Compression**: | ||
| ```python | ||
| import gzip | ||
| import json | ||
FrigaZzz marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| def _create_compressed_sse(self, payload): | ||
| json_str = json.dumps(payload, separators=(',', ':')) | ||
| compressed = gzip.compress(json_str.encode()) | ||
| # Use binary SSE or base64 encoding | ||
| return f"data: {base64.b64encode(compressed).decode()}\n\n" | ||
| ``` | ||
|
|
||
| 3. **Event Deduplication**: | ||
| ```python | ||
| class DedupSSEMapper(SSEEventMapper): | ||
| def __init__(self): | ||
| self._last_payloads = {} | ||
|
|
||
| def map_event_to_sse_message(self, event, level): | ||
| payload = super().map_event_to_sse_message(event, level) | ||
|
|
||
| # Skip if identical to last payload for this session | ||
| session_key = f"{event.session_id}_{event.author}" | ||
| if self._last_payloads.get(session_key) == payload: | ||
| return None | ||
|
|
||
| self._last_payloads[session_key] = payload | ||
| return payload | ||
| ``` | ||
|
|
||
|
|
||
| ## π€ Contributing | ||
|
|
||
| This template is designed to be extended and customized for your specific needs. Key extension points: | ||
|
|
||
| 1. **Router Classes**: Add domain-specific endpoints | ||
| 2. **SSE Mappers**: Custom event processing and optimization | ||
| 3. **Middleware**: Cross-cutting concerns | ||
| 4. **Services**: Additional business logic services | ||
| 5. **Configuration**: Environment-specific settings | ||
|
|
||
| ## π Further Resources | ||
|
|
||
| - **Google ADK Documentation**: https://google.github.io/adk-docs/ | ||
| - **FastAPI Documentation**: https://fastapi.tiangolo.com/ | ||
| - **Pydantic Settings**: https://docs.pydantic.dev/latest/concepts/pydantic_settings/ | ||
| - **Server-Sent Events**: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events | ||
|
|
||
| --- | ||
|
|
||
| **Happy coding!** π This template provides a solid foundation for building production-ready ADK extensions with modern Python patterns and performance optimizations. | ||
Empty file.
1 change: 1 addition & 0 deletions
1
contributing/samples/fastapi_modular_server/app/agents/greetings_agent/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| from app.agents.greetings_agent.greetings_agent import root_agent |
9 changes: 9 additions & 0 deletions
9
contributing/samples/fastapi_modular_server/app/agents/greetings_agent/greetings_agent.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| from google.adk.agents import LlmAgent | ||
|
|
||
| root_agent = LlmAgent( | ||
| model="gemini-2.5-flash", | ||
| name="greetings_agent", | ||
| description="A friendly Google Gemini-powered agent", | ||
| instruction="You are a helpful AI assistant powered by Google Gemini.", | ||
| tools=[], | ||
| ) |
Empty file.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.