Required Google Cloud Platform permissions seem to have changed

[itsthejb]

My operating system is (include version):

Cent OS 7

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

• Docker image (https://github.com/itsthejb/dockerfiles/blob/master/certbot/Dockerfile)

I ran this command and it produced this output:

certbot --dns-google --dns-google-credentials /etc/letsencrypt/credentials/credentials.json --server https://acme-v02.api.letsencrypt.org/directory renewal --dry-run (or similar)

• Also discussion at https://community.letsencrypt.org/t/dns-google-plugin-failure-to-find-managed-domains/88373/6

Certbot's behavior differed from what I expected because:

Attempting refresh to obtain initial access_token Refreshing access_token Encountered 403 Forbidden with reason "forbidden" Cleaning up challenges URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest URL being requested: GET https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net. Attempting refresh to obtain initial access_token Refreshing access_token Encountered 403 Forbidden with reason "forbidden" Error finding zone. Skipping cleanup. Attempting to renew cert (jcrooke.net) from /etc/letsencrypt/renewal/jcrooke.net.conf produced an unexpected error: Encountered error finding managed zone: <HttpError 403 when requesting https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net. returned "Forbidden">. Skipping.

• Tried with new service account as documented at https://certbot-dns-google.readthedocs.io/en/stable/#credentials. These permissions generate 403
• Tried also DNS Administrator Role. Also did not work
• Finally succeeded when using fully permission Owner role
• So it appears that something changed on the API, and the documented permissions are insufficient
• Since I'm not running a critical operation, I don't mind temporarily using a Power User for Certbot. This should probably be investigated and docs updated, however 😃

Here is a Certbot log showing the issue (if available):

2019-03-12 10:14:07,179:DEBUG:certbot.main:certbot version: 0.31.0
2019-03-12 10:14:07,179:DEBUG:certbot.main:Arguments: ['--dns-google', '--dns-google-credentials', '/etc/letsencrypt/credentials/credentials.json', '--server', 'https://acme-v02.api.letsencrypt.org/directory', '--dry-run']
2019-03-12 10:14:07,179:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#dns-google,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-03-12 10:14:07,197:DEBUG:certbot.log:Root logging level set at 20
2019-03-12 10:14:07,197:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-03-12 10:14:07,227:DEBUG:certbot.plugins.selection:Requested authenticator dns-google and installer <certbot.cli._Default object at 0x7f4778d45a50>
2019-03-12 10:14:07,227:DEBUG:certbot.cli:Var server=https://acme-v02.api.letsencrypt.org/directory (set by user).
2019-03-12 10:14:07,227:DEBUG:certbot.cli:Var authenticator=dns-google (set by user).
2019-03-12 10:14:07,254:INFO:certbot.renewal:Cert not due for renewal, but simulating renewal for dry run
2019-03-12 10:14:07,254:DEBUG:certbot.plugins.selection:Requested authenticator dns-google and installer None
2019-03-12 10:14:07,260:DEBUG:certbot.plugins.selection:Single candidate plugin: * dns-google
Description: Obtain certificates using a DNS TXT record (if you are using Google Cloud DNS for DNS).
Interfaces: IAuthenticator, IPlugin
Entry point: dns-google = certbot_dns_google.dns_google:Authenticator
Initialized: <certbot_dns_google.dns_google.Authenticator object at 0x7f4778d48810>
Prep: True
2019-03-12 10:14:07,261:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_dns_google.dns_google.Authenticator object at 0x7f4778d48810> and installer None
2019-03-12 10:14:07,261:INFO:certbot.plugins.selection:Plugins selected: Authenticator dns-google, Installer None
2019-03-12 10:14:07,263:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(status=None, terms_of_service_agreed=None, agreement=None, only_return_existing=None, contact=(), key=None, external_account_binding=None), uri=u'https://acme-staging-v02.api.letsencrypt.org/acme/acct/7251889', new_authzr_uri=None, terms_of_service=None), 6f625e748d8a79646a9eb3d97c8d9541, Meta(creation_host=u'48cac13b0bf5', creation_dt=datetime.datetime(2018, 10, 31, 17, 21, 59, tzinfo=<UTC>)))>
2019-03-12 10:14:07,264:DEBUG:acme.client:Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
2019-03-12 10:14:07,265:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org:443
2019-03-12 10:14:07,589:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 724
2019-03-12 10:14:07,589:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 724
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Tue, 12 Mar 2019 10:14:07 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Tue, 12 Mar 2019 10:14:07 GMT
Connection: keep-alive

{
  "dh-QUKoD0ZM": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}
2019-03-12 10:14:07,590:INFO:certbot.main:Renewing an existing certificate
2019-03-12 10:14:07,776:DEBUG:acme.client:Requesting fresh nonce
2019-03-12 10:14:07,778:DEBUG:acme.client:Sending HEAD request to https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce.
2019-03-12 10:14:07,977:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2019-03-12 10:14:07,977:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Link: <https://acme-staging-v02.api.letsencrypt.org/index>;rel="index"
Replay-Nonce: gqJw56FZJeUrmHo0Zh0kRa4P-Qjz1w7-f5VKB3pYaKA
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Content-Length: 0
Expires: Tue, 12 Mar 2019 10:14:07 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Tue, 12 Mar 2019 10:14:07 GMT
Connection: keep-alive


2019-03-12 10:14:07,978:DEBUG:acme.client:Storing nonce: gqJw56FZJeUrmHo0Zh0kRa4P-Qjz1w7-f5VKB3pYaKA
2019-03-12 10:14:07,978:DEBUG:acme.client:JWS payload:
{
  "identifiers": [
    {
      "type": "dns",
      "value": "*.jcrooke.net"
    }
  ]
}
2019-03-12 10:14:07,979:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "eyJub25jZSI6ICJncUp3NTZGWkplVXJtSG8wWmgwa1JhNFAtUWp6MXc3LWY1VktCM3BZYUtBIiwgInVybCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL25ldy1vcmRlciIsICJraWQiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hY2N0LzcyNTE4ODkiLCAiYWxnIjogIlJTMjU2In0",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwgCiAgICAgICJ2YWx1ZSI6ICIqLmpjcm9va2UubmV0IgogICAgfQogIF0KfQ",
  "signature": "t-ZoJOxkIvaSqIcPLm9-b26fHmc1ZZ9apPbuS314GpWkDFlISl2KAIKCZp8Ha782Y1rG4kfnqFl1Q-WtXSxhXtaleRqrGkc8hqILRySVw9Bd_UYDd3DL1TWN19EHj3qLSYg__5eWebghYc98ptvo4jNDaFs0aEff5wwMZB7yv07Sm4jIKSzu0VBQhqqvoOgXP9NjDHABUCzfbshBx-6uiK2Lzl4hAQyp-c5U2TJffzl1XONvrvxWv2qhEWPCZbWsszNTSk-OJY5LGMFLOfkkICRBJu-q1wXVupicuYKnQXcuMaskN9hnCbTbRgaelcRyVfU7PxUsV2daMfHBfejhEA"
}
2019-03-12 10:14:08,211:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 376
2019-03-12 10:14:08,212:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Content-Type: application/json
Content-Length: 376
Boulder-Requester: 7251889
Link: <https://acme-staging-v02.api.letsencrypt.org/index>;rel="index"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/order/7251889/26383659
Replay-Nonce: OgJDs4iYIFhDt3TpIN-FHEBe1rPQL3H0pFVtqEiITNs
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Tue, 12 Mar 2019 10:14:08 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Tue, 12 Mar 2019 10:14:08 GMT
Connection: keep-alive

{
  "status": "pending",
  "expires": "2019-03-17T22:06:05Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "*.jcrooke.net"
    }
  ],
  "authorizations": [
    "https://acme-staging-v02.api.letsencrypt.org/acme/authz/DmQgDt0VU_1Hrda_oqCXxqGyENwqnAKgS08aMUfDNzY"
  ],
  "finalize": "https://acme-staging-v02.api.letsencrypt.org/acme/finalize/7251889/26383659"
}
2019-03-12 10:14:08,212:DEBUG:acme.client:Storing nonce: OgJDs4iYIFhDt3TpIN-FHEBe1rPQL3H0pFVtqEiITNs
2019-03-12 10:14:08,212:DEBUG:acme.client:JWS payload:

2019-03-12 10:14:08,213:DEBUG:acme.client:Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz/DmQgDt0VU_1Hrda_oqCXxqGyENwqnAKgS08aMUfDNzY:
{
  "protected": "eyJub25jZSI6ICJPZ0pEczRpWUlGaER0M1RwSU4tRkhFQmUxclBRTDNIMHBGVnRxRWlJVE5zIiwgInVybCI6ICJodHRwczovL2FjbWUtc3RhZ2luZy12MDIuYXBpLmxldHNlbmNyeXB0Lm9yZy9hY21lL2F1dGh6L0RtUWdEdDBWVV8xSHJkYV9vcUNYeHFHeUVOd3FuQUtnUzA4YU1VZkROelkiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC83MjUxODg5IiwgImFsZyI6ICJSUzI1NiJ9",
  "payload": "",
  "signature": "CdNiBCP_bYFEnUPIr1Y4fOXou4xwEakocrpOToetkf2x6_x2aMv8_nuf3FNkKPerfz5clYyk2o03g7RaZ42OL6wkroA0dyn0rhW4h49m6H51Otr-VpqQwNTWxnyZXbg_YlfcNRu5lpyp3ojfVUpf_Lii2Xdmxe2GujIcRjH2Vxg4CY0_vGOcIbFvcyGix2aJAs6wtNWaWQKwzUZrGlp4Hn-8G_Ns6K-6pV_5WU4yXouSeZnglC5htwxfFCBGL6iyrlj1SeVsIT8gTaLuEt_wDT63yVsrEpcEiotE5t83YEu4dBzxDtA1FUbEthiJWI5ULAN8qAZ5xrpXmNgjBw_RBA"
}
2019-03-12 10:14:08,420:DEBUG:urllib3.connectionpool:https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz/DmQgDt0VU_1Hrda_oqCXxqGyENwqnAKgS08aMUfDNzY HTTP/1.1" 200 428
2019-03-12 10:14:08,421:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 428
Boulder-Requester: 7251889
Link: <https://acme-staging-v02.api.letsencrypt.org/index>;rel="index"
Replay-Nonce: zj7datbeVmSrvUThE1MGJArKvFd3PBhz-NP_JsrrzHc
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Tue, 12 Mar 2019 10:14:08 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Tue, 12 Mar 2019 10:14:08 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "jcrooke.net"
  },
  "status": "pending",
  "expires": "2019-03-17T22:06:05Z",
  "challenges": [
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/challenge/DmQgDt0VU_1Hrda_oqCXxqGyENwqnAKgS08aMUfDNzY/266575362",
      "token": "tMwFY1e7Y6BaalWNVfnZdHRN_gnjlbQFyUu4ECtruTY"
    }
  ],
  "wildcard": true
}
2019-03-12 10:14:08,421:DEBUG:acme.client:Storing nonce: zj7datbeVmSrvUThE1MGJArKvFd3PBhz-NP_JsrrzHc
2019-03-12 10:14:08,421:INFO:certbot.auth_handler:Performing the following challenges:
2019-03-12 10:14:08,422:INFO:certbot.auth_handler:dns-01 challenge for jcrooke.net
2019-03-12 10:14:08,424:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
2019-03-12 10:14:08,627:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net.
2019-03-12 10:14:08,627:INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
2019-03-12 10:14:08,629:DEBUG:oauth2client.crypt:['eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjRlYjFjNzk5NDE3NTI2NmJhYWZjYzZjMzhkZWUwZTk3OWRlMmYxOTgifQ', 'eyJpc3MiOiJjZXJ0Ym90QHdlYi1zZXJ2ZXItMjA4ODE0LmlhbS5nc2VydmljZWFjY291bnQuY29tIiwic2NvcGUiOiJodHRwczovL3d3dy5nb29nbGVhcGlzLmNvbS9hdXRoL25kZXYuY2xvdWRkbnMucmVhZHdyaXRlIiwiYXVkIjoiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJleHAiOjE1NTIzODkyNDgsImlhdCI6MTU1MjM4NTY0OH0', 'hMyT4bu6-sQT3k0BTmuaCp-fZQ4RKZ2JZkkhULwacSENp42GG65hSXBwBMFZmVNnfszPG72lVQ46OD4fk9SytfLbX8DmT80L3djrpOReLK7v3dW6F0gO5UKoW5xpSgWPDKtg7suh6dTirk1dvBCyMakOeRM0yxyFpdbIRaOxV0__PEhx7CsfTxqBvLmTXfWrW19siXrqtg_au_n-hnW5Mn40EP7L-5_j7M35qg1lTr2KenRqdtGpD2TuAj4xl9BDzEDrpxsigFZ3uQaqCb7fMyg3zryhKgakakF_irm32kcoDm763r50cGP1Ysci8IiBqasswqIU1jv-TevU44ySOg']
2019-03-12 10:14:08,629:INFO:oauth2client.client:Refreshing access_token
2019-03-12 10:14:09,222:WARNING:googleapiclient.http:Encountered 403 Forbidden with reason "forbidden"
2019-03-12 10:14:09,223:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/auth_handler.py", line 75, in handle_authorizations
    resp = self._solve_challenges(aauthzrs)
  File "/opt/certbot/src/certbot/auth_handler.py", line 139, in _solve_challenges
    resp = self.auth.perform(all_achalls)
  File "/opt/certbot/src/certbot/plugins/dns_common.py", line 57, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/opt/certbot/src/certbot-dns-google/certbot_dns_google/dns_google.py", line 70, in _perform
    self._get_google_client().add_txt_record(domain, validation_name, validation, self.ttl)
  File "/opt/certbot/src/certbot-dns-google/certbot_dns_google/dns_google.py", line 113, in add_txt_record
    zone_id = self._find_managed_zone_id(domain)
  File "/opt/certbot/src/certbot-dns-google/certbot_dns_google/dns_google.py", line 275, in _find_managed_zone_id
    .format(e))
PluginError: Encountered error finding managed zone: <HttpError 403 when requesting https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net. returned "Forbidden">

2019-03-12 10:14:09,223:DEBUG:certbot.error_handler:Calling registered functions
2019-03-12 10:14:09,223:INFO:certbot.auth_handler:Cleaning up challenges
2019-03-12 10:14:09,225:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/dns/v1/rest
2019-03-12 10:14:09,414:INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net.
2019-03-12 10:14:09,414:INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
2019-03-12 10:14:09,416:DEBUG:oauth2client.crypt:['eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IjRlYjFjNzk5NDE3NTI2NmJhYWZjYzZjMzhkZWUwZTk3OWRlMmYxOTgifQ', 'eyJpc3MiOiJjZXJ0Ym90QHdlYi1zZXJ2ZXItMjA4ODE0LmlhbS5nc2VydmljZWFjY291bnQuY29tIiwic2NvcGUiOiJodHRwczovL3d3dy5nb29nbGVhcGlzLmNvbS9hdXRoL25kZXYuY2xvdWRkbnMucmVhZHdyaXRlIiwiYXVkIjoiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJleHAiOjE1NTIzODkyNDksImlhdCI6MTU1MjM4NTY0OX0', 'Nw12THNUZXnIeLLwKIxZYAmQVCwxRsfSN_d-alPYeHs-CBYnvN0aecCbnIbZChYH6rpr0t_JeNae0Eugn0I4HIcy0-dHWO-eSxnc-BdxTq_jMprCp51sL34PhW3mEboK0kAUEUMJk9mG4r6hJz9BUp-Pb9K6vHots7yAMvayvb4VERUVBNuY4LIZHZF-JfgftESzoHZrXt_JR9U_UQhxBEIH0wtVR_6mfMq6LpzZGmg65aDocQauOHhhZGsGZ22-ldLG7ZJdoHwc7t9KkoS7p9_faPwOENfpwMj7Bsbr2Qy44JCcLv8cABNUIp1rvpLgzzvUqy0-STfOlHfjbw19Xw']
2019-03-12 10:14:09,416:INFO:oauth2client.client:Refreshing access_token
2019-03-12 10:14:10,537:WARNING:googleapiclient.http:Encountered 403 Forbidden with reason "forbidden"
2019-03-12 10:14:10,537:WARNING:certbot_dns_google.dns_google:Error finding zone. Skipping cleanup.
2019-03-12 10:14:10,537:WARNING:certbot.renewal:Attempting to renew cert (jcrooke.net) from /etc/letsencrypt/renewal/jcrooke.net.conf produced an unexpected error: Encountered error finding managed zone: <HttpError 403 when requesting https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net. returned "Forbidden">. Skipping.
2019-03-12 10:14:10,538:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/renewal.py", line 452, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/certbot/src/certbot/main.py", line 1193, in renew_cert
    renewed_lineage = _get_and_save_cert(le_client, config, lineage=lineage)
  File "/opt/certbot/src/certbot/main.py", line 116, in _get_and_save_cert
    renewal.renew_cert(config, domains, le_client, lineage)
  File "/opt/certbot/src/certbot/renewal.py", line 310, in renew_cert
    new_cert, new_chain, new_key, _ = le_client.obtain_certificate(domains, new_key)
  File "/opt/certbot/src/certbot/client.py", line 353, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/src/certbot/client.py", line 389, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
  File "/opt/certbot/src/certbot/auth_handler.py", line 75, in handle_authorizations
    resp = self._solve_challenges(aauthzrs)
  File "/opt/certbot/src/certbot/auth_handler.py", line 139, in _solve_challenges
    resp = self.auth.perform(all_achalls)
  File "/opt/certbot/src/certbot/plugins/dns_common.py", line 57, in perform
    self._perform(domain, validation_domain_name, validation)
  File "/opt/certbot/src/certbot-dns-google/certbot_dns_google/dns_google.py", line 70, in _perform
    self._get_google_client().add_txt_record(domain, validation_name, validation, self.ttl)
  File "/opt/certbot/src/certbot-dns-google/certbot_dns_google/dns_google.py", line 113, in add_txt_record
    zone_id = self._find_managed_zone_id(domain)
  File "/opt/certbot/src/certbot-dns-google/certbot_dns_google/dns_google.py", line 275, in _find_managed_zone_id
    .format(e))
PluginError: Encountered error finding managed zone: <HttpError 403 when requesting https://www.googleapis.com/dns/v1/projects/web-server-208814/managedZones?alt=json&dnsName=jcrooke.net. returned "Forbidden">

2019-03-12 10:14:10,538:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2019-03-12 10:14:10,538:ERROR:certbot.renewal:  /etc/letsencrypt/live/jcrooke.net/fullchain.pem (failure)
2019-03-12 10:14:10,540:INFO:certbot.hooks:Running post-hook command: /etc/letsencrypt/renewal-hooks/post/docker.sh
2019-03-12 10:14:18,213:INFO:certbot.hooks:Output from docker.sh:
ispconfig

2019-03-12 10:14:18,213:INFO:certbot.hooks:Running post-hook command: touch /etc/letsencrypt/renewed
2019-03-12 10:14:18,216:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 11, in <module>
    load_entry_point('certbot', 'console_scripts', 'certbot')()
  File "/opt/certbot/src/certbot/main.py", line 1365, in main
    return config.func(config, plugins)
  File "/opt/certbot/src/certbot/main.py", line 1272, in renew
    renewal.handle_renewal_request(config)
  File "/opt/certbot/src/certbot/renewal.py", line 477, in handle_renewal_request
    len(renew_failures), len(parse_failures)))
Error: 1 renew failure(s), 0 parse failure(s)

[schoen]

@joohoi, are you able to look at this? I imagine @zjs is usually too busy at the moment.

[xenorites]

I am encountering a similar issue, and I don't think the problem has to do with permissions. When running from GCE, the dns-google plugin only seems to examine the project the instance is running in for DNS zones. In your case @itsthejb, is the DNS zone "jcrooke.net" in the project "web-server-208814"?

I have two separate projects, one for my web application, one for managing all my DNS, and my web application is unable to request certs from the DNS zone. I have tried updating the SDKs running project with gcloud config set project 'dns-proj', however that seems to have no impact on which project certbot calls out to for the API.

[xenorites]

I was able to resolve my issue using the --dns-google-credentials and a service account from the DNS project. This, however feels unintuitive; I should be able to use the service account I assigned to the VM with the appropriate credentials to be able to do this. Things like gsutil have no problems between projects, as long as the correct permissions are assigned to the machine's service account.

For the record, I have created a custom permissions role with the exact permissions listed here.

[itsthejb]

Hi @xenorites,

I'm not running from GCE, and only have a single project. Plus I recreated all my service accounts. In my case, the elevated permission account is working around the issue for me, so seems clear I need some other permissions added to a "DNS Admin" account in order for it to work. Just don't know which specifically

[FearlessHyena]

I had the same problem and fixed it by adding the https://www.googleapis.com/auth/ndev.clouddns.readwrite scope to the Cloud API Access scope for the VM as mentioned here

I'm using Terraform so basically adding it to service_account.scopes makes it work

Would be great if this was added to the official documentation since it's not that obvious

[itsthejb]

Thanks for your input @FearlessHyena. Your suggestion doesn't apply to my particular setup, but I tried again, giving all DNS permissions, and I think it's now working:

[FearlessHyena]

Thanks for the follow up @itsthejb
I might need that in case the sope I'm using suddenly stops working in future!

[ghost]

Thanks for your input @FearlessHyena. Your suggestion doesn't apply to my particular setup, but I tried again, giving all DNS permissions, and I think it's now working:

Hi @FearlessHyena how do you get to this screenshot if i may ask. i

[FearlessHyena]

Hi @hatakora62 I think you meant to mention @itsthejb for the screenshot so just adding the mention here

Thanks for your input @FearlessHyena. Your suggestion doesn't apply to my particular setup, but I tried again, giving all DNS permissions, and I think it's now working:

Hi @FearlessHyena how do you get to this screenshot if i may ask. i

[itsthejb]

Glad I was helpful! Going to close this now, I’ve actually migrated away from certbot