feat: refactor debugtelemetry - optimize trace access

[pawel-maciejczek]

This PR

• refactors the signals storage to stop using test otel packages
• refactors the signals storage for efficient retrieval without iterating over all spans on each get request

We can add TTL to clean up old traces in following PR.

[gemini-code-assist]

Summary of Changes

Hello @pawel-maciejczek, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the debug telemetry service by centralizing the storage and retrieval of OpenTelemetry spans and logs. The primary goal is to provide a more complete and accurate view of telemetry data by ensuring that when a session ID is queried, all related spans within the entire trace are returned. This change improves the diagnostic capabilities by offering a holistic perspective of operations tied to a specific user session.

Highlights

• Unified Telemetry Storage: Refactored the DebugTelemetry service to use a single signalsHolder struct for managing both OpenTelemetry spans and logs, replacing separate in-memory exporters.
• Enhanced Session-Based Trace Retrieval: Modified GetSpansBySessionID to return all spans associated with a given session ID, including all spans within the same trace, rather than just individual spans directly tagged with the session ID.
• Improved Data Indexing: Introduced internal indexing within signalsHolder to efficiently map session IDs to trace IDs, and trace IDs to their respective spans, facilitating comprehensive trace retrieval.
• Test Coverage Expansion: Added new test cases to thoroughly validate the new functionality, ensuring that all relevant spans are correctly retrieved when querying by session ID, even across complex trace structures.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• server/adkrest/internal/services/debugtelemetry.go
  • Refactored DebugTelemetry to use a new signalsHolder for managing spans and logs.
  • Removed direct usage of tracetest.InMemoryExporter and InMemoryLogExporter.
  • Implemented sortAndConvertSignals to order and transform collected signals into DebugSpan objects.
  • Added new types signalHolder, inMemorySpan, and signalsHolder to encapsulate telemetry data and indexing logic.
  • Updated SpanProcessor and LogProcessor methods to integrate with the new signalsHolder.
  • Introduced ExportSpans and updateSpanIndexes methods within signalsHolder to build comprehensive trace and session mappings.
• server/adkrest/internal/services/debugtelemetry_test.go
  • Updated an existing test case to reflect the new behavior of GetSpansBySessionID.
  • Added new test cases, single_trace_all_returned and multi_trace_same_session, to verify the comprehensive retrieval of spans based on session ID across different trace scenarios.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the debug telemetry service to allow retrieving all spans within a trace for a given session ID, a significant enhancement from returning just a single span. The implementation introduces a new signalsHolder to unify span and log processing. My review focuses on a potential panic due to unsafe sorting, a data precision issue with timestamps, and a minor code clarity improvement in the tests. Overall, the changes are well-structured and the new functionality is well-tested.

[pawel-maciejczek]

/gemini review

[gemini-code-assist]

Code Review

This pull request refactors the debug telemetry service to improve performance and remove dependencies on test packages. The new spanStore implementation uses indexed maps for efficient retrieval of traces, which is a great improvement over iterating through all spans. The switch to batch processors for spans and logs also enhances performance. The code is well-structured and thread-safe. I have one minor suggestion to reduce code duplication.

[dpasiukevich]

Left a couple of comments, if these look ok, let's fix and it should be good to merge.

And could I ask not to force push and rather just apply commit on top if possible?
It's a lot easier to see the recent changes without reading the whole PR again.

[pawel-maciejczek]

Done. Sorry for the force pushes.

[dpasiukevich]

LGTM!