Skip to content

feat(agent): emit metainfo download time and throughput#596

Merged
Anton-Kalpakchiev merged 2 commits into
masterfrom
emit-metainfo-download-latency-and-throughput
Apr 16, 2026
Merged

feat(agent): emit metainfo download time and throughput#596
Anton-Kalpakchiev merged 2 commits into
masterfrom
emit-metainfo-download-latency-and-throughput

Conversation

@Anton-Kalpakchiev
Copy link
Copy Markdown
Collaborator

@Anton-Kalpakchiev Anton-Kalpakchiev commented Apr 16, 2026

Before every P2P download in an agent, the agent must first acquire the metainfo of that torrent. That process has shown to sometimes be a massive bottleneck in P2P downloads, as origin must download the whole blob from remote storage (e.g. GCS) in order to calculate its metainfo.

This PR adds client-side metrics (i.e. in agent) to record the latency and throughput of the metainfo request. Metrics are tagged by blob size, just like e2e torrent downloads in agent are.

Before this PR, the metainfo latency was already emitted, but

  1. that latency was not tagged by torrent_size (which is relevant to put the latency into context)
  2. the throughput of the download was not measured
  3. a timer was used, instead of a histogram, with timers emitting much more unneeded metrics compared to a histogram. Across a huge fleet of agents, this makes a big difference, so I am now using a histogram for efficiency.

Note: we also emit metainfo download latency server-side in tracker, but we cannot rely on that metric, as the tracker has an nginx cache in front of the application code, therefore emitting metrics only on cache misses, which provides a skewed distribution.

Copy link
Copy Markdown
Collaborator

@sambhav-jain-16 sambhav-jain-16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment

Comment thread lib/torrent/observability/download_performance.go
@Anton-Kalpakchiev Anton-Kalpakchiev merged commit 4e99cab into master Apr 16, 2026
10 checks passed
@Anton-Kalpakchiev Anton-Kalpakchiev deleted the emit-metainfo-download-latency-and-throughput branch April 16, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants