Skip to content

[fix] Truncate large query profiles after MergedProfile - #4034

Merged
morningman merged 1 commit into
apache:masterfrom
Ragnaroos:codex/profile-large-file-truncation
Aug 5, 2026
Merged

[fix] Truncate large query profiles after MergedProfile#4034
morningman merged 1 commit into
apache:masterfrom
Ragnaroos:codex/profile-large-file-truncation

Conversation

@Ragnaroos

Copy link
Copy Markdown
Contributor

Summary

  • accept local Query Profile files up to 100 MiB
  • reduce files over 10 MiB before upload by retaining Summary and MergedProfile
  • remove per-instance DetailProfile/Execution Profile sections
  • keep the final upload within the existing 10 MiB backend limit
  • prevent stale asynchronous file preparation from replacing a newer selection
  • add focused tests for truncation boundaries, UTF-8 byte size, and file limits

Why

Large Apache Doris Query Profiles can exceed the backend's 10 MiB upload limit because they contain extensive per-instance DetailProfile data.

Summary and MergedProfile retain the aggregated execution metrics needed for the initial diagnosis. Removing the following per-instance sections allows large Profiles to use the existing analysis API without increasing the backend request limit.

Truncation is only performed when a MergedProfile header is followed by a recognized per-instance profile header. Files that cannot be reduced safely to 10 MiB are rejected.

Validation

  • node --test src/components/profile-analysis/*.test.js
    • 51 tests passed
  • targeted TypeScript compilation passed
  • verified with a 41,070,035-byte real Profile
    • output: 123,362 bytes
    • MergedProfile retained
    • DetailProfile removed

@Ragnaroos
Ragnaroos marked this pull request as ready for review August 4, 2026 11:19
@morningman
morningman merged commit 119862e into apache:master Aug 5, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants