Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcp_compute plugin does not work #44404

Closed
jesusch opened this issue Aug 20, 2018 · 18 comments · Fixed by #50183
Closed

gcp_compute plugin does not work #44404

jesusch opened this issue Aug 20, 2018 · 18 comments · Fixed by #50183
Assignees
Labels
affects_2.6 This issue/PR affects Ansible v2.6 bug This issue/PR relates to a bug. needs_info This issue requires further information. Please answer any outstanding questions. support:community This issue/PR relates to code supported by the Ansible community.

Comments

@jesusch
Copy link

jesusch commented Aug 20, 2018

SUMMARY

gcp_compute plugin does not work. always an error that no matching inventory plugin is working

ISSUE TYPE
  • Bug Report
COMPONENT NAME

gcp_compute

ANSIBLE VERSION
ansible 2.6.3
  config file = /Users/jesusch/.ansible.cfg
  configured module search path = [u'/Users/jesusch/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15 (default, Jul 23 2018, 21:27:06) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
CONFIGURATION

DEFAULT_HOST_LIST(/Users/jesusch/.ansible.cfg) = [u'/Users/jesusch/ansible/inventory']
DEFAULT_ROLES_PATH(/Users/jesusch/.ansible.cfg) = [u'/Users/jesusch/ansible/roles']
HOST_KEY_CHECKING(/Users/jesusch/.ansible.cfg) = False
RETRY_FILES_ENABLED(/Users/jesusch/.ansible.cfg) = False

OS / ENVIRONMENT

Darwin Bjorns-MacBook-Pro.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64

STEPS TO REPRODUCE

Configured GCP plugin as described in https://docs.ansible.com/ansible/2.6/scenario_guides/guide_gce.html

---
plugin: gcp_compute
projects:
  - google.com:myProject-1337
filters: null
auth_kind: serviceaccount
service_account_file: /Users/jesusch/Downloads/myProject-1337-somehash.json
EXPECTED RESULTS

ansible-inventory --list -i inventory/myproject.gcp.yml should list my GC hosts and its metadata

ACTUAL RESULTS
ANSIBLE_DEBUG=1 ansible-inventory --list -i inventory/myproject.gcp.yml
 40804 1534774235.58981: starting run
 40804 1534774235.68904: Added group all to inventory
 40804 1534774235.68913: Added group ungrouped to inventory
 40804 1534774235.68919: Group all now contains ungrouped
 40804 1534774235.70007: Loading InventoryModule 'host_list' from /usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/host_list.py
 40804 1534774235.70106: assigned :doc
 40804 1534774235.70117: assigned :plainexamples
 40804 1534774235.70164: Loading InventoryModule 'script' from /usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/script.py
 40804 1534774235.70398: assigned :doc
 40804 1534774235.70418: Loaded config def from plugin (inventory/script)
 40804 1534774235.70446: Loading InventoryModule 'yaml' from /usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/yaml.py
 40804 1534774235.70576: assigned :doc
 40804 1534774235.70584: assigned :plainexamples
 40804 1534774235.70603: Loaded config def from plugin (inventory/yaml)
 40804 1534774235.70668: Loading InventoryModule 'ini' from /usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/ini.py
 40804 1534774235.70865: assigned :doc
 40804 1534774235.70874: assigned :plainexamples
 40804 1534774235.70911: Loading InventoryModule 'auto' from /usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/auto.py
 40804 1534774235.70965: assigned :doc
 40804 1534774235.70971: assigned :plainexamples
 40804 1534774235.70984: Examining possible inventory source: /Users/jesusch/git/ansible/inventory/myproject.gcp.yml
 40804 1534774235.70992: Attempting to use plugin host_list (/usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/host_list.py)
 40804 1534774235.70998: /Users/jesusch/git/ansible/inventory/myproject.gcp.yml did not meet host_list requirements
 40804 1534774235.71004: Attempting to use plugin script (/usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/script.py)
 40804 1534774235.71764: /Users/jesusch/git/ansible/inventory/myproject.gcp.yml was not parsable by script
 40804 1534774235.71790: Attempting to use plugin yaml (/usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/yaml.py)
 40804 1534774235.72000: Loading data from /Users/jesusch/git/ansible/inventory/myproject.gcp.yml
 40804 1534774235.72153: /Users/jesusch/git/ansible/inventory/myproject.gcp.yml was not parsable by yaml
 40804 1534774235.72162: Attempting to use plugin ini (/usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/ini.py)
 40804 1534774235.72316: set inventory_file for ---
 40804 1534774235.72330: set inventory_dir for ---
 40804 1534774235.72335: Added host --- to inventory
 40804 1534774235.72343: Added host --- to group ungrouped
 40804 1534774235.72386: /Users/jesusch/git/ansible/inventory/myproject.gcp.yml was not parsable by ini
 40804 1534774235.72393: Attempting to use plugin auto (/usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/auto.py)
 40804 1534774235.72421: Loading data from /Users/jesusch/git/ansible/inventory/myproject.gcp.yml
 40804 1534774235.77925: Loading InventoryModule 'gcp_compute' from /usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/gcp_compute.py
 40804 1534774235.79347: assigned :doc
 40804 1534774235.79357: assigned :plainexamples
 40804 1534774235.79887: Loading ModuleDocFragment 'constructed' from /usr/local/lib/python2.7/site-packages/ansible/utils/module_docs_fragments/constructed.py
 40804 1534774235.79947: Loading ModuleDocFragment 'inventory_cache' from /usr/local/lib/python2.7/site-packages/ansible/utils/module_docs_fragments/inventory_cache.py
 40804 1534774235.80017: Loaded config def from plugin (inventory/gcp_compute)
 40804 1534774235.80042: Loading data from /Users/jesusch/git/ansible/inventory/myproject.gcp.yml
 40804 1534774236.20005: auto failed to parse /Users/jesusch/git/ansible/inventory/myproject.gcp.yml
@ansibot
Copy link
Contributor

ansibot commented Aug 20, 2018

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibot ansibot added affects_2.6 This issue/PR affects Ansible v2.6 bug This issue/PR relates to a bug. needs_triage Needs a first human triage before being processed. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Aug 20, 2018
@webknjaz webknjaz removed the needs_triage Needs a first human triage before being processed. label Aug 21, 2018
@apettinen
Copy link

apettinen commented Aug 23, 2018

I hit this same error, but got it working via the following steps:

  • enable the plugin in your ansible.cfg (most inventory plugins are off by default), e.g.:
[inventory]
enable_plugins = gcp_compute, host_list, yaml, ini, script

see: https://docs.ansible.com/ansible/2.5/plugins/inventory.html for more details
Note, this also specifies the search order.

plugin: gcp_compute
zones: # populate inventory with instances in these regions
  - us-east1-a
projects:
  - gcp-prod-gke-100
  - gcp-cicd-101
filters:
  - machineType = n1-standard-1
  - scheduling.automaticRestart = true AND machineType = n1-standard-1
scopes:
  - https://www.googleapis.com/auth/compute
service_account_file: /tmp/service_account.json
auth_kind: serviceaccount

Also ensure your service account has enough permissions to access the desired resources.

This is on ansible 2.6.3 with python 2.7.10.
So at least the docs should be updated to mention the inventory plugins (or at least link to the docs).

@jesusch
Copy link
Author

jesusch commented Aug 29, 2018

Also explicit specifing the enable_plugins sort order does not do the trick
see:

 61631 1535544664.51556: Considering mycloud.gcp.yml
 61631 1535544664.51567: Examining possible inventory source: /Users/jesusch/ansible/inventory/mycloud.gcp.yml
 61631 1535544664.51578: Attempting to use plugin gcp_compute (/usr/local/lib/python2.7/site-packages/ansible/plugins/inventory/gcp_compute.py)
 61631 1535544664.51610: Loading data from /Users/jesusch/ansible/inventory/mycloud.gcp.yml
 61631 1535544664.78010: gcp_compute failed to parse /Users/jesusch/ansible/inventory/mycloud.gcp.yml
 61631 1535544664.79291: /Users/jesusch/ansible/inventory/mycloud.gcp.yml was not parsable by script
 [WARNING]:  * Failed to parse /Users/jesusch/ansible/inventory/mycloud.gcp.yml with gcp_compute plugin: 'NoneType' object has no attribute '__getitem__'
 61631 1535544664.79513: parsed /Users/jesusch/ansible/inventory/mycloud.gcp.yml as False

@ansibot ansibot added support:community This issue/PR relates to code supported by the Ansible community. and removed support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Sep 14, 2018
@jesusch
Copy link
Author

jesusch commented Sep 25, 2018

still an issue with 2.6.4

@jesusch
Copy link
Author

jesusch commented Oct 26, 2018

still an issue with 2.7.0

@bhundven
Copy link
Contributor

bhundven commented Oct 26, 2018

Issue #45105 is a duplicate of this issue.

I have also tested with 2.7.1, and the issue persists.

@walterdolce
Copy link

walterdolce commented Oct 31, 2018

I am also having this problem. Adding the [inventory] config section didn't work. Ansible version is 2.7.0.

@walterdolce
Copy link

walterdolce commented Oct 31, 2018

Quick update. For me the issue turned out to be I was trying to use a variable inside the dynamic inventory file.

This did not work because it was causing an "internal" Ansible error, not exposed even if ANSIBLE_DEBUG was set:

---
plugin: gcp_compute
# ...
service_account_file: "{{ ansible_env.PWD }}/.gcp/svca-key.json"
auth_kind: serviceaccount
# ...

And this wasn't working either, for the same reason:

---
plugin: gcp_compute
# ...
service_account_file: "{{ some_group_var_I_defined_containing_the_path_to_the_svc_key }}"
auth_kind: serviceaccount
# ...

@bcoca @bhundven Is it expected that Ansible does not compile/expand/parse variables defined in the GCP dynamic inventory and/or "built-in" variables?

@bhundven
Copy link
Contributor

So in your ansible.cfg you need to enable the gcp_compute plugin:

[inventory]
# enable inventory plugins, default: 'host_list', 'script', 'yaml', 'ini'
#enable_plugins = host_list, virtualbox, yaml, constructed
enable_plugins = host_list, script, yaml, ini, gcp_compute

This fixed this plugin for me.

@jesusch
Copy link
Author

jesusch commented Nov 1, 2018

also does not work for me:

  2896 1541067684.30548: gcp_compute failed to parse /Users/jesusch/ansible/inventory/wywy.gcp.yml
 [WARNING]:  * Failed to parse /Users/jesusch/ansible/inventory/wywy.gcp.yml with ini plugin: /Users/jesusch/ansible/inventory/wywy.gcp.yml:2: Expected key=value host variable assignment, got:
gcp_compute

 [WARNING]:  * Failed to parse /Users/jesusch/ansible/inventory/wywy.gcp.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory

 [WARNING]:  * Failed to parse /Users/jesusch/ansible/inventory/wywy.gcp.yml with constructed plugin: Incorrect plugin name in file: gcp_compute

 [WARNING]:  * Failed to parse /Users/jesusch/ansible/inventory/wywy.gcp.yml with gcp_compute plugin: 'NoneType' object has no attribute '__getitem__'

 [WARNING]: Unable to parse /Users/jesusch/ansible/inventory/wywy.gcp.yml as an inventory source

  2896 1541067684.30767: parsed /Users/jesusch/ansible/inventory/wywy.gcp.yml as False

@chad-autry
Copy link

The example config on the ansible page is misleading or incorrect. Remove "google.com:" just need the project identifier

---
plugin: gcp_compute
projects:
  - myProject-1337
...

@jesusch
Copy link
Author

jesusch commented Nov 12, 2018

@chad-autry this finally did the trick

documentation update should do the trick

Akasurde added a commit to Akasurde/ansible that referenced this issue Dec 20, 2018
* Update example about dynamic inventory
* minor typo fixes in gcp_utils
* Additional information about enabling inventory plugin in ansible.cfg

partially fixes: ansible#44404

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
@Akasurde
Copy link
Member

@jesusch @chad-autry Can you confirm if #50183 works for you ? Thanks.

needs_info

@ansibot ansibot added the needs_info This issue requires further information. Please answer any outstanding questions. label Dec 20, 2018
@jesusch
Copy link
Author

jesusch commented Dec 21, 2018

@Akasurde works for me :)

@jesusch jesusch closed this as completed Dec 21, 2018
@sudomann
Copy link

sudomann commented Dec 25, 2018

Such conflicting documentation;
docs here give the wrong format for project names:

projects:
  - google.com:graphite-playground

I got it to work with the following (notice no google.com:):

plugin: gcp_compute
projects:
  - my-gcp-project-id
filters:
scopes:
  - https://www.googleapis.com/auth/compute
service_account_file: /path/to/ansible-inventory-plugin-service-account.json
auth_kind: serviceaccount

This was based off other replies I saw on issue related to this, and the docs here which has it properly labelled.

@Akasurde
Copy link
Member

@sudomann PR #50183 is yet to merge. That is why you are seeing this conflicts in documentation. Once PR is merged, this conflict will go away.

drewmullen pushed a commit to drewmullen/ansible that referenced this issue Jan 3, 2019
* gcp: documentation update

* Update example about dynamic inventory
* minor typo fixes in gcp_utils
* Additional information about enabling inventory plugin in ansible.cfg

partially fixes: ansible#44404

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
kbreit pushed a commit to kbreit/ansible that referenced this issue Jan 11, 2019
* gcp: documentation update

* Update example about dynamic inventory
* minor typo fixes in gcp_utils
* Additional information about enabling inventory plugin in ansible.cfg

partially fixes: ansible#44404

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
@jeremyarr
Copy link

also another gotcha: the file must have a suffix gcp.yml

@Akasurde
Copy link
Member

@jeremyarr This is already mentioned in documentation - https://docs.ansible.com/ansible/latest/plugins/inventory/gcp_compute.html#synopsis

This was referenced Apr 9, 2019
@ansible ansible locked and limited conversation to collaborators Jul 22, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
affects_2.6 This issue/PR affects Ansible v2.6 bug This issue/PR relates to a bug. needs_info This issue requires further information. Please answer any outstanding questions. support:community This issue/PR relates to code supported by the Ansible community.
Projects
None yet
Development

Successfully merging a pull request may close this issue.