Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCP Cloud Run Service resource causing an error #8528

Closed
nvarscar opened this issue May 1, 2023 · 0 comments · Fixed by #8569
Closed

GCP Cloud Run Service resource causing an error #8528

nvarscar opened this issue May 1, 2023 · 0 comments · Fixed by #8569
Labels

Comments

@nvarscar
Copy link

nvarscar commented May 1, 2023

Describe the bug

When I run a policy that involves gcp.cloud-run-service resource it fails with an error

What did you expect to happen?

No error, policy processed

Cloud Provider

Google Cloud (GCP)

Cloud Custodian version and dependency information

Custodian:   0.9.26
Python:      3.10.10 (main, Feb 16 2023, 02:49:39) [Clang 14.0.0 (clang-1400.0.29.202)]
Platform:    posix.uname_result(sysname='Darwin', nodename='NA-23N99WXMBP1.local', release='22.4.0', version='Darwin Kernel Version 22.4.0: Mon Mar  6 20:59:28 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T6000', machine='arm64')
Using venv:  True
Docker: False
Installed: 

PyJWT==2.6.0
adal==1.2.7
applicationinsights==0.11.10
apscheduler==3.10.1
argcomplete==3.0.5
attrs==22.2.0
azure-common==1.1.28
azure-core==1.26.4
azure-cosmos==3.2.0
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.6
azure-functions==1.13.3
azure-graphrbac==0.61.1
azure-identity==1.12.0
azure-keyvault==4.2.0
azure-keyvault-certificates==4.7.0
azure-keyvault-keys==4.8.0
azure-keyvault-secrets==4.7.0
azure-mgmt-advisor==9.0.0
azure-mgmt-apimanagement==1.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-authorization==1.0.0
azure-mgmt-batch==15.0.0
azure-mgmt-cdn==10.0.0
azure-mgmt-cognitiveservices==11.0.0
azure-mgmt-compute==19.0.0
azure-mgmt-containerinstance==7.0.0
azure-mgmt-containerregistry==8.0.0b1
azure-mgmt-containerservice==15.1.0
azure-mgmt-core==1.4.0
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-costmanagement==1.0.0
azure-mgmt-databricks==1.0.0b1
azure-mgmt-datafactory==1.1.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-dns==8.0.0b1
azure-mgmt-eventgrid==8.0.0
azure-mgmt-eventhub==8.0.0
azure-mgmt-frontdoor==1.0.1
azure-mgmt-hdinsight==7.0.0
azure-mgmt-iothub==1.0.0
azure-mgmt-keyvault==8.0.0
azure-mgmt-logic==9.0.0
azure-mgmt-managementgroups==1.0.0b1
azure-mgmt-monitor==2.0.0
azure-mgmt-msi==1.0.0
azure-mgmt-network==17.1.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-policyinsights==1.0.0
azure-mgmt-rdbms==8.1.0
azure-mgmt-redis==12.0.0
azure-mgmt-resource==16.1.0
azure-mgmt-resourcegraph==7.0.0
azure-mgmt-search==8.0.0
azure-mgmt-security==1.0.0
azure-mgmt-servicefabric==1.0.0
azure-mgmt-sql==1.0.0
azure-mgmt-storage==17.1.0
azure-mgmt-subscription==1.0.0
azure-mgmt-trafficmanager==0.51.0
azure-mgmt-web==2.0.0
azure-nspkg==3.0.2
azure-storage-blob==12.15.0
azure-storage-common==2.1.0
azure-storage-file==2.1.0
azure-storage-file-share==12.11.1
azure-storage-queue==12.6.0
boto3==1.26.109
botocore==1.29.109
c7n==0.9.26
cachetools==5.3.0
certifi==2022.12.7
cffi==1.15.1
charset-normalizer==3.1.0
click==8.1.3
cryptography==40.0.1
distlib==0.3.6
docutils==0.18.1
google-api-core==2.11.0
google-api-python-client==2.84.0
google-auth==2.17.2
google-auth-httplib2==0.1.0
google-cloud-appengine-logging==1.3.0
google-cloud-audit-log==0.2.5
google-cloud-core==2.3.2
google-cloud-logging==3.5.0
google-cloud-monitoring==2.14.2
google-cloud-storage==2.8.0
google-crc32c==1.5.0
google-resumable-media==2.4.1
googleapis-common-protos==1.59.0
grpc-google-iam-v1==0.12.6
grpcio==1.53.0
grpcio-status==1.53.0
httplib2==0.22.0
idna==3.4
importlib-metadata==5.2.0
isodate==0.6.1
jmespath==1.0.1
jsonschema==4.17.3
msal==1.21.0
msal-extensions==1.0.0
msrest==0.7.1
msrestazure==0.6.4
netaddr==0.7.20
oauthlib==3.2.2
portalocker==2.7.0
proto-plus==1.22.2
protobuf==4.22.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.21
pyjwt==2.6.0
pyparsing==3.0.9
pyrate-limiter==2.10.0
pyrsistent==0.19.3
python-dateutil==2.8.2
pytz==2023.3
pytz-deprecation-shim==0.1.0.post0
pyyaml==6.0
requests==2.28.2
requests-oauthlib==1.3.1
retrying==1.3.4
rsa==4.9
s3transfer==0.6.0
setuptools==67.6.1
six==1.16.0
tabulate==0.9.0
typing-extensions==4.5.0
tzdata==2023.3
tzlocal==4.3
uritemplate==4.1.1
urllib3==1.26.15
zipp==3.15.0

Policy

policies:
  - name: ensure_gcp_instance_labels
    description: |
      Report resources without labels
    resource: gcp.cloud-run-service
    filters:
      - type: value
        key: labels."my-label"
        value: absent
    actions:
      - type: notify
        slack_template: slack_alert
        slack_msg_color: danger
        to:
          - slack://#custodian-reports
        transport:
          type: pubsub
          topic: projects/foobar/topics/custodian

Relevant log/traceback output

2023-04-27 17:16:37,650 - custodian.output - ERROR - Error while executing policy
Traceback (most recent call last):
  File "/Users/kirill.kravtsov/Library/Caches/pypoetry/virtualenvs/asset-hound-smtHBYHl-py3.10/lib/python3.10/site-packages/c7n/policy.py", line 331, in run
    resources = self.policy.resource_manager.resources()
  File "/Users/kirill.kravtsov/Library/Caches/pypoetry/virtualenvs/asset-hound-smtHBYHl-py3.10/lib/python3.10/site-packages/c7n_gcp/query.py", line 214, in resources
    resources = self.filter_resources(resources)
  File "/Users/kirill.kravtsov/Library/Caches/pypoetry/virtualenvs/asset-hound-smtHBYHl-py3.10/lib/python3.10/site-packages/c7n/manager.py", line 112, in filter_resources
    resources = f.process(resources, event)
  File "/Users/kirill.kravtsov/Library/Caches/pypoetry/virtualenvs/asset-hound-smtHBYHl-py3.10/lib/python3.10/site-packages/c7n/filters/core.py", line 339, in process
    return self.process_set(resources, event)
  File "/Users/kirill.kravtsov/Library/Caches/pypoetry/virtualenvs/asset-hound-smtHBYHl-py3.10/lib/python3.10/site-packages/c7n/filters/core.py", line 356, in process_set
    resource_map = {r[rtype_id]: r for r in resources}
  File "/Users/kirill.kravtsov/Library/Caches/pypoetry/virtualenvs/asset-hound-smtHBYHl-py3.10/lib/python3.10/site-packages/c7n/filters/core.py", line 356, in <dictcomp>
    resource_map = {r[rtype_id]: r for r in resources}
KeyError: 'name'

Extra information or context

The issue is related to the resource definition:

https://github.com/cloud-custodian/cloud-custodian/blob/main/tools/c7n_gcp/c7n_gcp/resources/cloudrun.py#L19

The actual object doesn't have an id/name fields and should probably be:

        name = "metadata.name"
        id = "metadata.uid"

I can confirm that after changing this code the resource starts working as intended. I can reasonably infer that CloudRunJob class will experience similar issues, but I don't have any to test my theory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant