Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Ensure topic exists in consumer only if autoCreateResources is true (#2294) #2296

Merged
merged 1 commit into from
Oct 26, 2023

Conversation

burkedavison
Copy link
Member

Fix #2294.

PR #2233 minimized the topic permissions requirement on consumers when autoCreateResources=false, but also changed the existing behavior of creating the topic when autoCreateResources=true. This PR reinstates the "create topics in consumers when autoCreateResources=true" while also avoiding any topic checks (and thus maintaining minimal permissions requirements) when autoCreateResources=false.

CC @SheheryarAamir

@burkedavison burkedavison requested a review from a team as a code owner October 26, 2023 14:09
@sonarcloud
Copy link

sonarcloud bot commented Oct 26, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@burkedavison
Copy link
Member Author

Test failure due to (unrelated) quota limit on BigQuery table operations:
https://github.com/GoogleCloudPlatform/spring-cloud-gcp/actions/runs/6655345599/job/18085411171

Error:  com.google.cloud.spring.bigquery.integration.outbound.BigQueryFileMessageHandlerIntegrationTests.testLoadFile_cancel -- Time elapsed: 4.530 s <<< ERROR!
com.google.cloud.bigquery.BigQueryException: Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:114)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.deleteTable(HttpBigQueryRpc.java:357)
	at com.google.cloud.bigquery.BigQueryImpl$10.call(BigQueryImpl.java:592)
	at com.google.cloud.bigquery.BigQueryImpl$10.call(BigQueryImpl.java:589)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.bigquery.BigQueryImpl.delete(BigQueryImpl.java:588)
	at com.google.cloud.spring.bigquery.integration.outbound.BigQueryFileMessageHandlerIntegrationTests.setup(BigQueryFileMessageHandlerIntegrationTests.java:74)
...
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
DELETE https://www.googleapis.com/bigquery/v2/projects/spring-cloud-gcp-ci/datasets/test_dataset/tables/test_table
***
  "code": 403,
  "errors": [
    ***
      "domain": "usageLimits",
      "message": "Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
      "reason": "rateLimitExceeded"
    ***
  ],
  "message": "Exceeded rate limits: too many table update operations for this table. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas",
  "status": "PERMISSION_DENIED"
***

@burkedavison burkedavison merged commit c8d25c8 into main Oct 26, 2023
42 checks passed
@burkedavison burkedavison deleted the pubsub-ensure-topic-only-if-autoCreateResources branch October 26, 2023 16:06
burkedavison added a commit to burkedavison/spring-cloud-gcp that referenced this pull request Oct 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PubSubChannelProvisioner no longer auto creates topics for consumers
2 participants