Skip to content

Remove telemetry API data limits and refactor URL builders#16020

Merged
JamesNK merged 3 commits intomainfrom
jamesnk/telemetry-api-limits
Apr 10, 2026
Merged

Remove telemetry API data limits and refactor URL builders#16020
JamesNK merged 3 commits intomainfrom
jamesnk/telemetry-api-limits

Conversation

@JamesNK
Copy link
Copy Markdown
Member

@JamesNK JamesNK commented Apr 9, 2026

Description

Remove artificial telemetry data limits in the dashboard API and CLI, and refactor URL builder methods for type safety and consistency.

Changes:

  • Remove MaxQueryCount from TelemetryApiService: Storage queries now use int.MaxValue so all data is returned and limits are controlled by the caller's limit parameter.
  • Add TelemetryCommandHelpers.MaxTelemetryLimit (int.MaxValue): CLI export and MCP tools pass this to fetch all available telemetry data in a single API call.
  • Refactor DashboardUrls telemetry URL builders: Changed from params (string, string?)[] tuples to explicit named parameters matching MapTelemetryApi endpoints, improving type safety and discoverability.
  • Replace BuildQueryString with AddQueryString pattern: Aligns telemetry URL construction with the existing pattern used by StructuredLogsUrl and other URL builders in the same class.
  • Add tests: Limit-returns-most-recent tests for spans, traces, and logs; large-limit test for 20k logs; updated URL builder tests.

Motivation: The export command and MCP tools were silently truncating telemetry data due to default limits (100 traces, 200 logs) when no limit parameter was passed. This made exports incomplete without any warning.

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
    • No
  • Does the change require an update in our Aspire docs?
    • Yes
    • No

JamesNK added 2 commits April 10, 2026 06:16
- Remove MaxQueryCount from TelemetryApiService, use int.MaxValue so
  storage queries return all data and limits are controlled by callers.
- Add TelemetryCommandHelpers.MaxTelemetryLimit (int.MaxValue) for CLI
  export and MCP tools to fetch all available telemetry data.
- Refactor DashboardUrls telemetry URL builders from params tuples to
  explicit named parameters matching MapTelemetryApi endpoints.
- Replace BuildQueryString with AddQueryString pattern for consistency.
- Update tests for new URL builder signatures and large limit scenarios.
Add GetSpans_WithLimit_ReturnsMostRecentSpans,
GetTraces_WithLimit_ReturnsMostRecentTraces, and
GetLogs_WithLimit_ReturnsMostRecentLogs tests to TelemetryApiServiceTests.
Copilot AI review requested due to automatic review settings April 9, 2026 22:26
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 16020

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 16020"

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes artificial data caps in the dashboard telemetry API/CLI flows (so exports and MCP tools don’t silently truncate), and refactors telemetry URL builders in DashboardUrls to use explicit, endpoint-aligned parameters.

Changes:

  • Updated dashboard telemetry API queries to request all stored items (Count = int.MaxValue) and rely on the caller-provided limit for truncation.
  • Added a CLI-side “fetch max available” limit constant and propagated it to export + MCP telemetry fetches.
  • Refactored DashboardUrls telemetry URL builders to explicit parameters and aligned query-string building with the existing AddQueryString pattern; updated/added unit and integration tests.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/Shared/Telemetry/TelemetryTestHelpers.cs Adds maxLogCount option to allow large-log scenarios in tests.
tests/Aspire.Dashboard.Tests/TelemetryApiServiceTests.cs Adds coverage for “limit returns most recent” and large-limit log retrieval.
tests/Aspire.Dashboard.Tests/Integration/TelemetryApiTests.cs Updates integration coverage to use very large limit values in query-param checks.
tests/Aspire.Cli.Tests/Commands/TelemetryCommandTests.cs Updates tests to validate new telemetry logs URL builder behavior.
src/Shared/DashboardUrls.cs Refactors telemetry URL builders to explicit parameters and uses AddQueryString + shared resource param appending.
src/Aspire.Dashboard/Api/TelemetryApiService.cs Removes MaxQueryCount and requests all stored telemetry items from storage before applying API-level limit.
src/Aspire.Cli/Mcp/Tools/ListTraceStructuredLogsTool.cs Ensures MCP trace structured logs fetch requests the maximum available data.
src/Aspire.Cli/Mcp/Tools/ListTracesTool.cs Ensures MCP traces fetch requests the maximum available data.
src/Aspire.Cli/Mcp/Tools/ListStructuredLogsTool.cs Ensures MCP structured logs fetch requests the maximum available data.
src/Aspire.Cli/Commands/TelemetryTracesCommand.cs Simplifies URL construction using new typed URL builder parameters.
src/Aspire.Cli/Commands/TelemetrySpansCommand.cs Simplifies URL construction and preserves follow-mode limit behavior via typed URL builder parameters.
src/Aspire.Cli/Commands/TelemetryLogsCommand.cs Simplifies URL construction and preserves follow-mode limit behavior via typed URL builder parameters.
src/Aspire.Cli/Commands/TelemetryCommandHelpers.cs Adds MaxTelemetryLimit constant (used to request “max available” results).
src/Aspire.Cli/Commands/ExportCommand.cs Ensures export fetches all available structured logs/traces in a single API call.
Comments suppressed due to low confidence (2)

src/Aspire.Dashboard/Api/TelemetryApiService.cs:56

  • When multiple resources are specified, this method queries the repository once per resource key and then aggregates results with AddRange. Traces/spans that involve more than one of the selected resources can be returned multiple times, inflating TotalCount/ReturnedCount and potentially causing duplicates in the response. Consider deduplicating by (TraceId, SpanId) after aggregation or changing the query strategy to avoid per-resource queries for multi-resource filters.
        foreach (var resourceKey in resourceKeys)
        {
            var result = telemetryRepository.GetTraces(new GetTracesRequest
            {
                ResourceKey = resourceKey,
                StartIndex = 0,
                Count = int.MaxValue,
                Filters = [],
                FilterText = string.Empty
            });
            allSpans.AddRange(result.PagedResult.Items.SelectMany(t => t.Spans));

src/Aspire.Dashboard/Api/TelemetryApiService.cs:127

  • For multi-resource filters, aggregating GetTraces results per resource key can duplicate the same trace when it contains spans from more than one selected resource. This can skew TotalCount/ReturnedCount and cause duplicates to be returned/limited incorrectly. Consider deduplicating by TraceId after aggregation (or avoid per-resource querying when multiple resource names are supplied).
        foreach (var resourceKey in resourceKeys)
        {
            var result = telemetryRepository.GetTraces(new GetTracesRequest
            {
                ResourceKey = resourceKey,
                StartIndex = 0,
                Count = int.MaxValue,
                Filters = [],
                FilterText = string.Empty
            });
            allTraces.AddRange(result.PagedResult.Items);

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

🎬 CLI E2E Test Recordings — 55 recordings uploaded (commit 264bf01)

View recordings
Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View Recording
AddPackageWhileAppHostRunningDetached ▶️ View Recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View Recording
AgentInitCommand_DefaultSelection_InstallsSkillOnly ▶️ View Recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View Recording
AllPublishMethodsBuildDockerImages ▶️ View Recording
AspireAddPackageVersionToDirectoryPackagesProps ▶️ View Recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View Recording
Banner_DisplayedOnFirstRun ▶️ View Recording
Banner_DisplayedWithExplicitFlag ▶️ View Recording
Banner_NotDisplayedWithNoLogoFlag ▶️ View Recording
CertificatesClean_RemovesCertificates ▶️ View Recording
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate ▶️ View Recording
CertificatesTrust_WithUntrustedCert_TrustsCertificate ▶️ View Recording
ConfigSetGet_CreatesNestedJsonFormat ▶️ View Recording
CreateAndRunAspireStarterProject ▶️ View Recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View Recording
CreateAndRunEmptyAppHostProject ▶️ View Recording
CreateAndRunJavaEmptyAppHostProject ▶️ View Recording
CreateAndRunJsReactProject ▶️ View Recording
CreateAndRunPythonReactProject ▶️ View Recording
CreateAndRunTypeScriptEmptyAppHostProject ▶️ View Recording
CreateAndRunTypeScriptStarterProject ▶️ View Recording
CreateJavaAppHostWithViteApp ▶️ View Recording
CreateStartAndStopAspireProject ▶️ View Recording
CreateTypeScriptAppHostWithViteApp ▶️ View Recording
DashboardRunWithOtelTracesReturnsNoTraces ▶️ View Recording
DescribeCommandResolvesReplicaNames ▶️ View Recording
DescribeCommandShowsRunningResources ▶️ View Recording
DetachFormatJsonProducesValidJson ▶️ View Recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View Recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View Recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View Recording
GlobalMigration_HandlesCommentsAndTrailingCommas ▶️ View Recording
GlobalMigration_HandlesMalformedLegacyJson ▶️ View Recording
GlobalMigration_PreservesAllValueTypes ▶️ View Recording
GlobalMigration_SkipsWhenNewConfigExists ▶️ View Recording
GlobalSettings_MigratedFromLegacyFormat ▶️ View Recording
InvalidAppHostPathWithComments_IsHealedOnRun ▶️ View Recording
LegacySettingsMigration_AdjustsRelativeAppHostPath ▶️ View Recording
LogsCommandShowsResourceLogs ▶️ View Recording
PsCommandListsRunningAppHost ▶️ View Recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View Recording
PublishWithDockerComposeServiceCallbackSucceeds ▶️ View Recording
RestoreGeneratesSdkFiles ▶️ View Recording
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes ▶️ View Recording
RunFromParentDirectory_UsesExistingConfigNearAppHost ▶️ View Recording
SecretCrudOnDotNetAppHost ▶️ View Recording
SecretCrudOnTypeScriptAppHost ▶️ View Recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View Recording
StopAllAppHostsFromAppHostDirectory ▶️ View Recording
StopAllAppHostsFromUnrelatedDirectory ▶️ View Recording
StopNonInteractiveMultipleAppHostsShowsError ▶️ View Recording
StopNonInteractiveSingleAppHost ▶️ View Recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View Recording

📹 Recordings uploaded automatically from CI run #24216472327

@JamesNK JamesNK merged commit d71bb01 into main Apr 10, 2026
265 checks passed
@JamesNK JamesNK deleted the jamesnk/telemetry-api-limits branch April 10, 2026 01:30
@JamesNK
Copy link
Copy Markdown
Member Author

JamesNK commented Apr 10, 2026

/backport to release/13.2

@github-actions
Copy link
Copy Markdown
Contributor

Started backporting to release/13.2 (link to workflow run)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants