Skip to content

fix(orch): denormalize upload metric file type#2865

Merged
ValentaTomas merged 2 commits into
mainfrom
add-upload-compression-metrics
May 30, 2026
Merged

fix(orch): denormalize upload metric file type#2865
ValentaTomas merged 2 commits into
mainfrom
add-upload-compression-metrics

Conversation

@ValentaTomas
Copy link
Copy Markdown
Member

Summary

Use one denormalized file_type metric attribute for upload compression metrics (memfile, memfile-header, rootfs, rootfs-header, snap, meta) instead of splitting artifact and file_type.

Test plan

  • cd packages/orchestrator && go test ./pkg/sandbox -run TestNonexistent
  • cd packages/orchestrator && golangci-lint run ./pkg/sandbox/...

@cla-bot cla-bot Bot added the cla-signed label May 29, 2026
@cursor
Copy link
Copy Markdown

cursor Bot commented May 29, 2026

PR Summary

Low Risk
Observability-only label and helper changes; upload behavior and storage paths are unchanged.

Overview
Upload compression telemetry now tags every upload with a single file_type dimension (memfile, memfile-header, rootfs, rootfs-header, snap, meta) instead of splitting artifact (data vs header) and a separate file kind. Snap and meta blob uploads go through uploadBlobWithMetrics, which records byte counts the same way as framed data and header stores.

Reviewed by Cursor Bugbot for commit 0d0b76b. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors sandbox upload metrics by replacing generic artifact and header identifiers with specific file type constants and introducing the uploadBlobWithMetrics helper to track snapshot and metadata uploads. There are no review comments, and I have no feedback to provide.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
2695 1 2694 7
View the full list of 3 ❄️ flaky test(s)
github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestSandboxListPaginationRunningLargerLimit

Flake rate in main: 42.83% (Passed 726 times, Failed 544 times)

Stack Traces | 92.8s run time
=== RUN   TestSandboxListPaginationRunningLargerLimit
    sandbox_list_test.go:327: Created sandbox 1/12: imshydgn8mdxo2nmjzurl
    sandbox_list_test.go:327: Created sandbox 2/12: i1thtljrm7woaunuswt6g
    sandbox_list_test.go:327: Created sandbox 3/12: imqmmycqmzo6v1m01dkh4
    sandbox_list_test.go:327: Created sandbox 4/12: iykf51wk33lu2dssskuhe
    sandbox_list_test.go:327: Created sandbox 5/12: iflvlx40tw2fd0y2npv5d
    sandbox_list_test.go:327: Created sandbox 6/12: i8qh5us96ku2qfzmcfnog
    sandbox_list_test.go:327: Created sandbox 7/12: ieua0wl6pr70orklwqhqo
    sandbox_list_test.go:327: Created sandbox 8/12: i0s736zbof97eq6rzoa1o
    sandbox_list_test.go:327: Created sandbox 9/12: i1ibvl1r7di5gf9oxo6p7
    sandbox_list_test.go:327: Created sandbox 10/12: iomahrm8jjrj5vzwx4s97
    sandbox_list_test.go:327: Created sandbox 11/12: iymyg4d26zivbi95nwl56
    sandbox_list_test.go:327: Created sandbox 12/12: ikfmves6gurol4el2klc2
    sandbox_list_test.go:330: 
        	Error Trace:	.../api/sandboxes/sandbox_list_test.go:340
        	            				.../hostedtoolcache/go/1.26.3.../src/runtime/asm_amd64.s:1771
        	Error:      	"[]" should have 12 item(s), but has 0
    sandbox_list_test.go:330: 
        	Error Trace:	.../api/sandboxes/sandbox_list_test.go:330
        	Error:      	Condition never satisfied
        	Test:       	TestSandboxListPaginationRunningLargerLimit
--- FAIL: TestSandboxListPaginationRunningLargerLimit (92.82s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity

Flake rate in main: 57.88% (Passed 719 times, Failed 988 times)

Stack Traces | 69.6s run time
=== RUN   TestSandboxMemoryIntegrity
=== PAUSE TestSandboxMemoryIntegrity
=== CONT  TestSandboxMemoryIntegrity
    sandbox_memory_integrity_test.go:27: Build completed successfully
--- FAIL: TestSandboxMemoryIntegrity (69.59s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity/tmpfs_hash

Flake rate in main: 58.00% (Passed 709 times, Failed 979 times)

Stack Traces | 200s run time
=== RUN   TestSandboxMemoryIntegrity/tmpfs_hash
=== PAUSE TestSandboxMemoryIntegrity/tmpfs_hash
=== CONT  TestSandboxMemoryIntegrity/tmpfs_hash
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{start:{pid:1265}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Total memory: 985 MB\nUsed memory before tmpfs mount: 188 MB\nFree memory before tmpfs mount: 796 MB\nMemory to use in integrity test (60% of free, min 64MB): 477 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"477+0 records in\n477+0 records out\n500170752 bytes (500 MB, 477 MiB) copied, 2.45631 s, 204 MB/s\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"\tCommand being timed: \"dd if=/dev/urandom of=/mnt/testfile bs=1M count=477\"\n\tUser time (seconds): 0.00\n\tSystem time (seconds): 2.44\n\tPercent of CPU this job got: 99%\n\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:02.46\n\tAverage shared text size (kbytes): 0\n\tAverage unshared data size (kbytes): 0\n\tAverage stack size (kbytes): 0\n\tAverage total size (kbytes): 0\n\tMaximum resident set size (kbytes): 2628\n\tAverage resident set size (kbytes): 0\n\tMajor (requiring I/O) page faults: 2\n\tMinor (reclaiming a frame) page faults: 342\n\tVoluntary context switches: 3\n\tInvoluntary context switches: 47\n\tSwaps: 0\n\tFile system inputs: 176\n\tFile system outputs: 0\n\tSocket messages sent: 0\n\tSocket messages received: 0\n\tSignals delivered: 0\n\tPage size (bytes): 4096\n\tExit status: 0\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Used memory after tmpfs mount and file fill: 670 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:70: Command [bash] completed successfully in sandbox il26v5k8sxurcwpbmyslz
Executing command bash in sandbox iwhb3ab8and9nzdqtxkll (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1281}}
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{data:{stdout:"877df45c68417b578620d2c3f655a091fc91ca758e445149858f562954989dc6\n"}}
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{end:{exited:true  status:"exit status 0"}}
    sandbox_memory_integrity_test.go:80: Command [bash] completed successfully in sandbox il26v5k8sxurcwpbmyslz
Executing command bash in sandbox iwhb3ab8and9nzdqtxkll (user: root)
    sandbox_memory_integrity_test.go:80: Command [bash] output: event:{start:{pid:1284}}
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
Executing command bash in sandbox il26v5k8sxurcwpbmyslz (user: root)
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:81
        	            				.../hostedtoolcache/go/1.26.3.../src/runtime/asm_amd64.s:1771
        	Error:      	Received unexpected error:
        	            	failed to execute command bash in sandbox il26v5k8sxurcwpbmyslz: unavailable: HTTP status 502 Bad Gateway
    sandbox_memory_integrity_test.go:110: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:78
        	            				.../tests/orchestrator/sandbox_memory_integrity_test.go:110
        	Error:      	Condition never satisfied
        	Test:       	TestSandboxMemoryIntegrity/tmpfs_hash
--- FAIL: TestSandboxMemoryIntegrity/tmpfs_hash (199.63s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: V4 rootfs metric emits "rootfs.ext4" instead of "rootfs"
    • Added mapping logic to convert fileType to the correct metric constant (uploadFileRootfs) before calling recordUploadCompression, ensuring V4 metrics emit 'rootfs' like V3.

Create PR

Or push these changes by commenting:

@cursor push 0203492d5b
Preview (0203492d5b)
diff --git a/packages/orchestrator/pkg/sandbox/build_upload_v4.go b/packages/orchestrator/pkg/sandbox/build_upload_v4.go
--- a/packages/orchestrator/pkg/sandbox/build_upload_v4.go
+++ b/packages/orchestrator/pkg/sandbox/build_upload_v4.go
@@ -93,7 +93,11 @@
 			compressedSize = size
 		}
 
-		recordUploadCompression(ctx, string(fileType), cfg, size, compressedSize)
+		metricFileType := uploadFileMemfile
+		if fileType == build.Rootfs {
+			metricFileType = uploadFileRootfs
+		}
+		recordUploadCompression(ctx, metricFileType, cfg, size, compressedSize)
 		selfBuild = headers.BuildData{Size: size, Checksum: checksum, FrameData: ft}
 	}

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit 1350fd8. Configure here.

Comment thread packages/orchestrator/pkg/sandbox/build_upload_v4.go Outdated
@ValentaTomas ValentaTomas marked this pull request as ready for review May 29, 2026 23:13
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1350fd8b17

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/orchestrator/pkg/sandbox/build_upload_v4.go Outdated
@ValentaTomas ValentaTomas merged commit b1646ca into main May 30, 2026
51 checks passed
@ValentaTomas ValentaTomas deleted the add-upload-compression-metrics branch May 30, 2026 22:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants