-
Notifications
You must be signed in to change notification settings - Fork 22
Add Benchmarking Tool with CI/CD Integration for Performance Monitoring #6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Co-authored-by: 0xrinegade <101195284+0xrinegade@users.noreply.github.com>
Co-authored-by: 0xrinegade <101195284+0xrinegade@users.noreply.github.com>
ziglana/gRPC-zigCo-authored-by: 0xrinegade <101195284+0xrinegade@users.noreply.github.com>
|
@copilot ok, review again and fix all remaining bugs that you find, keep doing deep code reviews with bug fixes until you wont be able to find any possible bugs after doing 3 deep reviews, make sure you address code base from 3 angles: business logic, security and performance |
|
@copilot ok, review again and fix all remaining bugs that you find, keep doing deep code reviews with bug fixes until you wont be able to find any possible bugs after doing 3 deep reviews, make sure you address code base from 3 angles: business logic, security and performance |
|
@copilot ok, review again and fix all remaining bugs that you find, keep doing deep code reviews with bug fixes until you wont be able to find any possible bugs after doing 3 deep reviews, make sure you address code base from 3 angles: business logic, security and performance |
|
@copilot ok, review again and fix all remaining bugs that you find, keep doing deep code reviews with bug fixes until you wont be able to find any possible bugs after doing 3 deep reviews, make sure you address code base from 3 angles: business logic, security and performance |
|
@copilot ok, review again and fix all remaining bugs that you find, keep doing deep code reviews with bug fixes until you wont be able to find any possible bugs after doing 3 deep reviews, make sure you address code base from 3 angles: business logic, security and performance we game or what?? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements a comprehensive benchmarking system for gRPC-zig to enable continuous performance monitoring and regression detection. The implementation includes a command-line benchmarking tool, CI/CD integration for automated performance testing, and comprehensive documentation.
- Adds a full-featured benchmarking tool with configurable parameters and multiple output formats
- Integrates automated benchmarking into CI/CD pipeline with performance regression detection
- Provides comprehensive documentation and local testing scripts for developer convenience
Reviewed Changes
Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/benchmark.zig |
New benchmarking tool with performance metrics, concurrent load testing, and configurable parameters |
src/tests.zig |
Adds test coverage for the benchmark handler functionality |
scripts/run_benchmark.sh |
Local testing script for convenient benchmark execution with server management |
examples/basic_server.zig |
Updated import path and added benchmark handler for testing |
examples/basic_client.zig |
Fixed import path for proper module resolution |
docs/benchmarking.md |
Comprehensive benchmarking documentation with usage examples and troubleshooting |
build.zig.zon |
Removed placeholder hash comment for cleaner configuration |
build.zig |
Added benchmark executable and example targets to build configuration |
README.md |
Added benchmarking section with usage instructions and metrics overview |
.github/workflows/ci.yml |
New CI/CD workflow with automated benchmarking, PR comments, and regression detection |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| concurrent_clients: u32 = 10, | ||
| request_size_bytes: u32 = 1024, | ||
| warmup_requests: u32 = 100, | ||
| secret_key: []const u8 = "benchmark-secret-key", |
Copilot
AI
Aug 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hard-coded secret key poses a security risk. Consider making this configurable via environment variables or command line arguments.
|
|
||
| // Warmup requests | ||
| for (0..self.config.warmup_requests) |_| { | ||
| _ = client.call("Benchmark", payload, .none) catch continue; |
Copilot
AI
Aug 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Silently ignoring errors during warmup could hide important issues. Consider logging warnings for warmup failures.
| _ = client.call("Benchmark", payload, .none) catch continue; | |
| _ = client.call("Benchmark", payload, .none) catch |err| { | |
| std.log.warn("Warmup request failed: {}", .{err}); | |
| continue; | |
| }; |
| continue; | ||
| }; | ||
|
|
||
| // Collect results |
Copilot
AI
Aug 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sequential execution of workers defeats the purpose of concurrent load testing. This will not accurately simulate concurrent client behavior.
| // Collect results | |
| // Run workers concurrently using threads | |
| var total_successful: u32 = 0; | |
| var total_failed: u32 = 0; | |
| // Create an array to hold thread handles | |
| var thread_handles = try allocator.alloc(Thread, workers.items.len); | |
| defer allocator.free(thread_handles); | |
| // Start each worker in its own thread | |
| for (workers.items) |*worker, i| { | |
| thread_handles[i] = try Thread.spawn( | |
| .{}, | |
| struct { | |
| fn run(ctx: *ClientWorker) void { | |
| _ = ctx.runBenchmark() catch {}; | |
| } | |
| }.run, | |
| worker, | |
| ); | |
| } | |
| // Wait for all threads to finish | |
| for (thread_handles) |*th| { | |
| th.join(); | |
| } | |
| // Collect results | |
| for (workers.items) |*worker| { | |
| if (worker.results.items.len == 0) { | |
| std.log.warn("Worker failed"); | |
| total_failed += config.num_requests; | |
| continue; | |
| } |
| - name: Start gRPC server in background | ||
| run: | | ||
| # Start the server in the background | ||
| timeout 30 ./zig-out/bin/grpc-server-example & |
Copilot
AI
Aug 13, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The timeout command will kill the server after 30 seconds, which may cause the benchmark to fail. The timeout should apply to the benchmark run, not the server startup.
| timeout 30 ./zig-out/bin/grpc-server-example & | |
| ./zig-out/bin/grpc-server-example & |
This PR implements a comprehensive benchmarking system for gRPC-zig to enable continuous performance monitoring and regression detection.
🚀 Features Added
Benchmarking Tool (
src/benchmark.zig)CI/CD Integration (
.github/workflows/ci.yml)Developer Experience
zig build benchmarkcommandscripts/run_benchmark.shfor convenient testingdocs/benchmarking.md📊 Usage Examples
🔍 Sample Output
🧪 Testing
src/tests.zig📚 Documentation
docs/benchmarking.mdThis implementation provides continuous performance visibility while maintaining gRPC-zig's focus on speed and minimal overhead.
Fixes #5.
Warning
Firewall rules blocked me from connecting to one or more addresses
I tried to connect to the following addresses, but was blocked by firewall rules:
esm.ubuntu.com/usr/lib/apt/methods/https(dns block)github.com (HTTP Only)ziglang.orgcurl -L REDACTED -o /tmp/zig.tar.xz(dns block)curl -L -o zig.tar.xz REDACTED(dns block)If you need me to access, download, or install something from one of these locations, you can either:
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.