Fixed bug where the method _find_zone_or_region cannot see some instances

[Ahuri3]

Fixed bug where _find_zone_or_region cannot see some instances

Description

The method _find_zone_or_region queries the google cloud api on the endpoint instances.aggregatedList.

This endpoint cannot return more than 500 results but has a pagination mechanic. The method _find_zone_or_region does not make use of the pagination mechanic because it made a manual request to the api.

I removed the manual request and replaced it by the method request_aggregated_items that was made for this exact purpose.

This bug was found while trying to use the libcloud in ansible (via the gce_snapshot ansible module) and getting an error libcloud.common.google.ResourceNotFoundError: "Node 'node_name' not found in any zone.".

Indeed without the pagination, if a google cloud project has more than 500 instances, the code will crash when trying to identify the zone for a gce instance that wasn't in the 500 instances returned by the first api call.

Here is the stack trace I saw :

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_qBlsq5/ansible_module_gce_snapshot.py", line 225, in <module>
    main()
  File "/tmp/ansible_qBlsq5/ansible_module_gce_snapshot.py", line 190, in main
    instance = gce.ex_get_node(instance_name, 'all')
  File "/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 7030, in ex_get_node
    name, 'instances', res_name='Node')
  File "/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py", line 7561, in _find_zone_or_region
    (res_name, name, rz), None, None)
libcloud.common.google.ResourceNotFoundError: "Node 'server-0504' not found in any zone."
fatal: [pr-op-cass00-0004 -> localhost]: FAILED! => {
    "changed": false, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_Gk7vXV/ansible_module_gce_snapshot.py\", line 225, in <module>\n    main()\n  File \"/tmp/ansible_Gk7vXV/ansible_module_gce_snapshot.py\", line 190, in main\n    instance = gce.ex_get_node(instance_name, 'all')\n  File \"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py\", line 7030, in ex_get_node\n    name, 'instances', res_name='Node')\n  File \"/var/lib/rundeck/.conda/envs/eva-prod/lib/python2.7/site-packages/libcloud/compute/drivers/gce.py\", line 7561, in _find_zone_or_region\n    (res_name, name, rz), None, None)\nlibcloud.common.google.ResourceNotFoundError: \"Node 'server-0504' not found in any zone.\"\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 1
}
...ignoring

Status

ready for review

Checklist (tick everything that applies)

• Code linting (required, can be done after the PR checks)
  • No pagination was changed
  • Line lenght is shorter than 79 characters
• Documentation
  • N/A
• Tests
  • Tested with Python 2.7.13 :: Continuum Analytics, Inc.
• ICLA (required for bigger changes)
  • N/A

[codecov-io]

Codecov Report

Merging #1203 into trunk will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##            trunk    #1203      +/-   ##
==========================================
+ Coverage   86.21%   86.22%   +<.01%     
==========================================
  Files         348      348              
  Lines       67382    67381       -1     
  Branches     5963     5963              
==========================================
+ Hits        58096    58097       +1     
+ Misses       6853     6851       -2     
  Partials     2433     2433

Impacted Files	Coverage Δ
libcloud/compute/drivers/gce.py	76.69% <100%> (-0.01%)	⬇️
libcloud/test/compute/test_ec2.py	97.93% <0%> (+0.15%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 694b021...4a7d884. Read the comment docs.

[pquentin]

Thanks! I'll try to merge this during the week. If not, please remind me.

[pquentin]

Thanks! Merged in trunk. ✨