-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pubsub: unit tests cause emulator to use excessive CPU #697
Comments
Thank you for the detailed analysis. That commit increases the number of worker goroutines from 1 to In your test, could you try adding |
Hi there - that's the workaround that we have in place for now. We still see a slow growth in CPU usage of 1-2% with each run. |
Ah, I'm sorry. I misread the part where you said "High CPU usage persists after the test suite has completed". I'll loop in the emulator folks. |
The fix for this has been made to the emulator and going to be published soon. |
This is an auto-generated regeneration of the gapic clients by cloud.google.com/go/internal/gapicgen. Once the corresponding genproto PR is submitted, genbot will update this PR with a newer dependency to the newer version of genproto and assign reviewers to this PR. If you have been assigned to review this PR, please: - Ensure that the version of genproto in go.mod has been updated. - Ensure that CI is passing. If it's failing, it requires your manual attention. - Approve and submit this PR if you believe it's ready to ship. Corresponding genproto PR: googleapis/go-genproto#742 Changes: feat(automl): publish updated protos for cloud/automl/v1 service fix!: One of the fields now have field_behavior as REQUIRED in cloud/automl/v1 service definition. Committer: @hkdevandla PiperOrigin-RevId: 421156406 Source-Link: googleapis/googleapis@05b24d8 fix(talent): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421134169 Source-Link: googleapis/googleapis@ef417dc fix(workflows): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098450 Source-Link: googleapis/googleapis@6706d9b fix(managedidentities): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098407 Source-Link: googleapis/googleapis@a635f71 fix(documentai): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098374 Source-Link: googleapis/googleapis@706ac2b fix(datalabeling): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098303 Source-Link: googleapis/googleapis@ac12f33 fix(apigateway): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098261 Source-Link: googleapis/googleapis@4c13e8f fix(asset): add service_yaml to php_gapic_library targets PiperOrigin-RevId: 421075193 Source-Link: googleapis/googleapis@dc15873 feat(compute): remove BETA language on Compute V1 (#697) Source-Link: googleapis/googleapis@b4b8eda fix(artifactregistry): add missing HTTP rules to service config PiperOrigin-RevId: 420790332 Source-Link: googleapis/googleapis@3965406
This is an auto-generated regeneration of the gapic clients by cloud.google.com/go/internal/gapicgen. Once the corresponding genproto PR is submitted, genbot will update this PR with a newer dependency to the newer version of genproto and assign reviewers to this PR. If you have been assigned to review this PR, please: - Ensure that the version of genproto in go.mod has been updated. - Ensure that CI is passing. If it's failing, it requires your manual attention. - Approve and submit this PR if you believe it's ready to ship. Corresponding genproto PR: googleapis/go-genproto#742 Changes: feat(automl): publish updated protos for cloud/automl/v1 service fix!: One of the fields now have field_behavior as REQUIRED in cloud/automl/v1 service definition. Committer: @hkdevandla PiperOrigin-RevId: 421156406 Source-Link: googleapis/googleapis@05b24d8 fix(talent): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421134169 Source-Link: googleapis/googleapis@ef417dc fix(workflows): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098450 Source-Link: googleapis/googleapis@6706d9b fix(managedidentities): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098407 Source-Link: googleapis/googleapis@a635f71 fix(documentai): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098374 Source-Link: googleapis/googleapis@706ac2b fix(datalabeling): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098303 Source-Link: googleapis/googleapis@ac12f33 fix(apigateway): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098261 Source-Link: googleapis/googleapis@4c13e8f fix(asset): add service_yaml to php_gapic_library targets PiperOrigin-RevId: 421075193 Source-Link: googleapis/googleapis@dc15873 feat(compute): remove BETA language on Compute V1 (#697) Source-Link: googleapis/googleapis@b4b8eda fix(artifactregistry): add missing HTTP rules to service config PiperOrigin-RevId: 420790332 Source-Link: googleapis/googleapis@3965406
This is an auto-generated regeneration of the gapic clients by cloud.google.com/go/internal/gapicgen. Once the corresponding genproto PR is submitted, genbot will update this PR with a newer dependency to the newer version of genproto and assign reviewers to this PR. If you have been assigned to review this PR, please: - Ensure that the version of genproto in go.mod has been updated. - Ensure that CI is passing. If it's failing, it requires your manual attention. - Approve and submit this PR if you believe it's ready to ship. Corresponding genproto PR: googleapis/go-genproto#742 Changes: feat(automl): publish updated protos for cloud/automl/v1 service fix!: One of the fields now have field_behavior as REQUIRED in cloud/automl/v1 service definition. Committer: @hkdevandla PiperOrigin-RevId: 421156406 Source-Link: googleapis/googleapis@05b24d8 fix(talent): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421134169 Source-Link: googleapis/googleapis@ef417dc fix(workflows): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098450 Source-Link: googleapis/googleapis@6706d9b fix(managedidentities): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098407 Source-Link: googleapis/googleapis@a635f71 fix(documentai): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098374 Source-Link: googleapis/googleapis@706ac2b fix(datalabeling): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098303 Source-Link: googleapis/googleapis@ac12f33 fix(apigateway): add ancillary service bindings to service_yaml PiperOrigin-RevId: 421098261 Source-Link: googleapis/googleapis@4c13e8f fix(asset): add service_yaml to php_gapic_library targets PiperOrigin-RevId: 421075193 Source-Link: googleapis/googleapis@dc15873 feat(compute): remove BETA language on Compute V1 (#697) Source-Link: googleapis/googleapis@b4b8eda fix(artifactregistry): add missing HTTP rules to service config PiperOrigin-RevId: 420790332 Source-Link: googleapis/googleapis@3965406
The pubsub emulator recently started using large amounts of CPU on OSX during our unit tests (other platforms untested). High CPU usage persists after the test suite has completed and all clients disconnected, until the process is restarted. I am able to reproduce this behaviour with simple clients and with google-cloud-go's own test suite.
Steps to reproduce:
gcloud beta emulators pubsub start --host-port=localhost:8822
cloud.google.com/go/pubsub
, run the test suite:env PUBSUB_EMULATOR_HOST=localhost:8822 go test -v
On my system, the pubsub emulator Java process CPU usage is at 45% after one run, 145% after the second run, and so forth.
This is undoubtedly a bug in the emulator, but git bisect shows that google-cloud-go only started triggering the issue with commit 927c812. This commit is included in the v0.10.0 release.
Software versions used:
The text was updated successfully, but these errors were encountered: