Add shutdown performance profiling summary#20911
Add shutdown performance profiling summary#20911dchen024 wants to merge 1 commit intogoogle-gemini:mainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the user experience by introducing detailed performance metrics upon session shutdown. Users can now gain valuable insights into the throughput, startup times, tool usage, and model API performance of their sessions, facilitating better understanding and optimization of their interactions with the CLI. The changes involve plumbing telemetry data through the application and rendering it in a user-friendly, structured format. Highlights
Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a performance summary on session shutdown, which is a great addition for providing developers with valuable insights. The implementation is solid, plumbing startup and wall-time metrics through to the UI and refactoring the tool profiling into a more compact and informative table. I've identified a couple of areas for improvement in the new StatsDisplay component to enhance the clarity and usefulness of the presented data. Specifically, I'm suggesting a change to how MCP tool names are displayed to avoid redundancy and to sort the model performance data to make it easier to interpret.
| ): { category: string; toolCall: string } => { | ||
| const tool = config.getToolRegistry()?.getTool(toolName); | ||
| if (tool instanceof DiscoveredMCPTool) { | ||
| return { category: tool.serverName, toolCall: toolName }; |
There was a problem hiding this comment.
For better clarity in the tool profiling summary, it's better to display the original tool name from the MCP server (serverToolName) instead of the potentially qualified toolName from the registry. This avoids redundancy where the category (server name) might be repeated in the tool call column.
| return { category: tool.serverName, toolCall: toolName }; | |
| return { category: tool.serverName, toolCall: tool.serverToolName }; |
| const modelPerformanceRows = Object.entries(models) | ||
| .filter(([, modelMetrics]) => modelMetrics.api.totalRequests > 0) | ||
| .map(([modelName, modelMetrics]) => { | ||
| const requests = modelMetrics.api.totalRequests; | ||
| const errors = modelMetrics.api.totalErrors; | ||
| const errorRate = requests > 0 ? (errors / requests) * 100 : 0; | ||
| const avgLatencyMs = | ||
| requests > 0 ? modelMetrics.api.totalLatencyMs / requests : 0; | ||
|
|
||
| return { | ||
| modelKey: modelName, | ||
| modelName: getDisplayString(modelName.replace('-001', '')), | ||
| requests, | ||
| errors, | ||
| errorRate, | ||
| avgLatencyMs, | ||
| }; | ||
| }); |
There was a problem hiding this comment.
The modelPerformanceRows array is not sorted. To improve the readability and usefulness of the "Model API Performance" summary, it would be beneficial to sort this data. Sorting by the number of requests in descending order would be a good approach, as it highlights the most frequently used models first.
| const modelPerformanceRows = Object.entries(models) | |
| .filter(([, modelMetrics]) => modelMetrics.api.totalRequests > 0) | |
| .map(([modelName, modelMetrics]) => { | |
| const requests = modelMetrics.api.totalRequests; | |
| const errors = modelMetrics.api.totalErrors; | |
| const errorRate = requests > 0 ? (errors / requests) * 100 : 0; | |
| const avgLatencyMs = | |
| requests > 0 ? modelMetrics.api.totalLatencyMs / requests : 0; | |
| return { | |
| modelKey: modelName, | |
| modelName: getDisplayString(modelName.replace('-001', '')), | |
| requests, | |
| errors, | |
| errorRate, | |
| avgLatencyMs, | |
| }; | |
| }); | |
| const modelPerformanceRows = Object.entries(models) | |
| .filter(([, modelMetrics]) => modelMetrics.api.totalRequests > 0) | |
| .map(([modelName, modelMetrics]) => { | |
| const requests = modelMetrics.api.totalRequests; | |
| const errors = modelMetrics.api.totalErrors; | |
| const errorRate = requests > 0 ? (errors / requests) * 100 : 0; | |
| const avgLatencyMs = | |
| requests > 0 ? modelMetrics.api.totalLatencyMs / requests : 0; | |
| return { | |
| modelKey: modelName, | |
| modelName: getDisplayString(modelName.replace('-001', '')), | |
| requests, | |
| errors, | |
| errorRate, | |
| avgLatencyMs, | |
| }; | |
| }) | |
| .sort((a, b) => b.requests - a.requests); |
gemini-cli
Bot
added
the
status/need-issue
|
Hi there! Thank you for your contribution to Gemini CLI. To improve our contribution process and better track changes, we now require all pull requests to be associated with an existing issue, as announced in our recent discussion and as detailed in our CONTRIBUTING.md. This pull request is being closed because it is not currently linked to an issue. Once you have updated the description of this PR to link an issue (e.g., by adding How to link an issue: Thank you for your understanding and for being a part of our community! |
1 participant
Summary
Example
Testing