Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce the number of times env.Processes gets called #482

Merged
merged 8 commits into from
Sep 28, 2023

Conversation

dhurley
Copy link
Collaborator

@dhurley dhurley commented Sep 22, 2023

Proposed changes

  • Only call env.Processes once on startup of the agent
  • Plugins that require process information now subscribe to core.NginxDetailProcUpdate to get any updates
  • The process watcher plugin now only checks for process changes every 5 seconds instead of every 500 milliseconds

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING document
  • I have run make install-tools and have attached any dependency changes to this pull request
  • If applicable, I have added tests that prove my fix is effective or that my feature works
  • If applicable, I have checked that any relevant tests pass after adding my changes
  • If applicable, I have updated any relevant documentation (README.md)
  • If applicable, I have tested my cross-platform changes on Ubuntu 22, Redhat 8, SUSE 15 and FreeBSD 13

@netlify
Copy link

netlify bot commented Sep 22, 2023

Deploy Preview for agent-public-docs canceled.

Name Link
🔨 Latest commit ff37049
🔍 Latest deploy log https://app.netlify.com/sites/agent-public-docs/deploys/6514430867157a0008b89ab5

@github-actions github-actions bot added chore Pull requests for routine tasks dependencies labels Sep 22, 2023
@codecov-commenter
Copy link

codecov-commenter commented Sep 22, 2023

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (d75b51e) 66.40% compared to head (ff37049) 66.25%.
Report is 1 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #482      +/-   ##
==========================================
- Coverage   66.40%   66.25%   -0.16%     
==========================================
  Files         118      118              
  Lines       13392    13412      +20     
==========================================
- Hits         8893     8886       -7     
- Misses       3907     3935      +28     
+ Partials      592      591       -1     
Files Coverage Δ
...r/github.com/nginx/agent/v2/src/plugins/metrics.go 68.06% <100.00%> (+1.13%) ⬆️
...dor/github.com/nginx/agent/v2/src/plugins/nginx.go 71.42% <100.00%> (-0.21%) ⬇️
....com/nginx/agent/v2/src/plugins/process_watcher.go 87.09% <100.00%> (-0.14%) ⬇️
...hub.com/nginx/agent/v2/src/plugins/registration.go 79.02% <100.00%> (+0.14%) ⬆️
...or/github.com/nginx/agent/v2/src/plugins/common.go 70.93% <88.88%> (+0.69%) ⬆️
...com/nginx/agent/v2/src/plugins/dataplane_status.go 67.28% <71.42%> (-0.48%) ⬇️
.../github.com/nginx/agent/v2/src/plugins/features.go 36.79% <45.45%> (-0.21%) ⬇️
...github.com/nginx/agent/v2/src/plugins/agent_api.go 61.93% <50.00%> (-0.87%) ⬇️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@oliveromahony
Copy link
Contributor

oliveromahony commented Sep 22, 2023

can you regenerate the profile.cgo?

  1. Run agent on a production system with pprof enabled (for about an hour, run through some of the workflows, metrics, config push etc).
  2. Take a cpu profile from the REST API (call it cpu.prof)
  3. Run the performance tests with cpu profiling:
test-bench: ## Run benchmark tests
	cd test/performance && GOWORK=off CGO_ENABLED=0 go test -mod=vendor -cpuprofile metrics_test_cpu.prof -count 5 -timeout 2m -bench=. metrics_test.go
	cd test/performance && GOWORK=off CGO_ENABLED=0 go test -mod=vendor -cpuprofile user_workflow_test_cpu.prof -count 1 -bench=. user_workflow_test.go
	cd test/performance && GOWORK=off CGO_ENABLED=0 go test -mod=vendor -cpuprofile plugins_test_cpu.prof -count 5 -timeout 2m -bench=. plugins_test.go
	cd test/performance && GOWORK=off CGO_ENABLED=0 go test -mod=vendor -cpuprofile environment_test_cpu.prof -count 5 -timeout 2m -bench=. environment_test.go
  1. Rename profile.cgo to profile.pprof
  2. Merge profile.pprof with cpu.prof, metrics_test_cpu.prof user_workflow_test_cpu.prof environment_test_cpu.prof metrics_test_cpu.prof
    go tool pprof -proto profile.pprof metrics_test_cpu.prof user_workflow_test_cpu.prof environment_test_cpu.prof metrics_test_cpu.prof > profile.cgo

@dhurley dhurley merged commit df8cf6e into main Sep 28, 2023
28 checks passed
@dhurley dhurley deleted the optimize-process-calls branch September 28, 2023 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Pull requests for routine tasks dependencies
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants