Skip to content

Stable upload progress and honest ETA in miren deploy#804

Merged
phinze merged 1 commit into
mainfrom
phinze/mir-1136-upload-progress-ETA
May 14, 2026
Merged

Stable upload progress and honest ETA in miren deploy#804
phinze merged 1 commit into
mainfrom
phinze/mir-1136-upload-progress-ETA

Conversation

@phinze
Copy link
Copy Markdown
Contributor

@phinze phinze commented May 14, 2026

While babysitting a slow deploy I noticed the progress line was lying to me. The "total" in Uploading artifacts: 0% — 24 KB / ~45.7 MB at 2.4 KB/s was swinging wildly: it started at ~974 KB, climbed steadily to ~80 MB over a few minutes, dropped back to ~20 MB, climbed again. Useless for figuring out how long the upload had left.

Tracing the math, the underlying bug was that EstimatedTotalBytes = BytesRead / Fraction was mixing two counters sampled on different sides of the gzip pipeline. written is uncompressed source bytes that have entered tar/gzip, while BytesRead is compressed bytes drained by the HTTP uploader on the other side of a gzip buffer plus io.Pipe. Under network back-pressure they advance in bursts at very different ratios, and their quotient is meaningless.

The fix is to stop pretending we know the compressed total. We don't, and projecting it from the runtime ratio doesn't work. The percentage itself was always fine, because both numerator and denominator are uncompressed bytes from the delta-filtered subset of files, so it correctly reports "fraction of the upload work consumed."

Since estimating remaining time is genuinely useful on slow uploads (the calculator-free "this will take 8 hours" use case), I also wove in a time-domain ETA: elapsed * (1 - frac) / frac. That formula sidesteps the unit-mixing trap entirely. There's a 5s warmup so early wild numbers don't show, and the display uses a tilde (eta ~8h 15m) to be upfront about being approximate.

The new line reads Uploading artifacts: 1% — 1.0 MB at 2.4 KB/s (eta ~8h 15m), in both explain mode and the TUI.

Closes MIR-1136

@phinze phinze requested a review from a team as a code owner May 14, 2026 00:07
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 40126e79-1643-4f92-879a-1326a6894381

📥 Commits

Reviewing files that changed from the base of the PR and between 77de783 and cde6863.

📒 Files selected for processing (4)
  • cli/commands/deploy.go
  • cli/commands/deploy_test.go
  • cli/commands/deploy_ui.go
  • pkg/progress/upload/upload.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • cli/commands/deploy_test.go
  • cli/commands/deploy.go
  • cli/commands/deploy_ui.go

📝 Walkthrough

Walkthrough

This PR changes upload progress reporting in the deploy CLI explain mode from displaying estimated total compressed bytes to displaying estimated time remaining (ETA). The Progress struct is updated to expose Fraction and ETA fields instead of EstimatedTotalBytes. The enrichUploadProgress function now computes these fields using uncompressed bytes written and elapsed-time extrapolation with warmup suppression. The UI model tracks uploadETA instead of uploadEstTotal, and the rendered progress line appends an ETA suffix when available. A new test validates enrichment behavior across zero-total, warmup, steady-state, and completion scenarios.


Comment @coderabbitai help to get the list of available commands and usage tips.

The "Uploading artifacts: N% — uploaded / ~TOTAL at SPEED" line had a
total estimate that wandered all over the place during slow uploads
(observed: ~974 KB initial, climbing to ~80 MB, periodically dropping
back to ~20 MB). The math was sampling uncompressed-bytes-in and
compressed-bytes-out on different sides of a gzip buffer plus io.Pipe,
so under network back-pressure their ratio was meaningless.

Drop the synthetic total entirely. The percentage still works because
both numerator (written) and denominator (totalUncompressed) only count
the delta-filtered subset of files.

Add an ETA extrapolated in the time domain: elapsed * (1 - frac) /
frac. This avoids the unit-mixing that broke the old projection. A 5s
warmup gates it so the first few wildly-fluctuating ticks do not show,
and the display uses "(eta ~Xh Ym)" with a tilde to telegraph that
it is approximate.

Closes MIR-1136
@phinze phinze force-pushed the phinze/mir-1136-upload-progress-ETA branch from 77de783 to cde6863 Compare May 14, 2026 00:11
Copy link
Copy Markdown
Contributor

@evanphx evanphx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image

@phinze phinze changed the title Stable upload progress and honest ETA in m deploy Stable upload progress and honest ETA in miren deploy May 14, 2026
@phinze phinze merged commit 201688a into main May 14, 2026
19 checks passed
@phinze phinze deleted the phinze/mir-1136-upload-progress-ETA branch May 14, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants