'NoneType' object is not iterable

[randyzwitch]

Just cloned the newest version of clusterdock, fresh start of Docker on OSX:

$ clusterdock_run ./bin/start_cluster -n testing cdh --primary-node=node-1 --secondary-nodes='node-{2..4}'
latest: Pulling from cloudera/clusterdock
Digest: sha256:8b125ab1c3c1edfaac9d6fd628586580bfcfa41cf72c5795151ed8e94c080bb2
Status: Image is up to date for cloudera/clusterdock:latest
Traceback (most recent call last):
  File "./bin/start_cluster", line 70, in <module>
    main()
  File "./bin/start_cluster", line 63, in main
    actions.start(args)
  File "/root/clusterdock/clusterdock/topologies/cdh/actions.py", line 54, in start
    if args.always_pull or not is_image_available_locally(image):
  File "/root/clusterdock/clusterdock/docker_utils.py", line 81, in is_image_available_locally
    return any(tag == name for image in client.images() for tag in image['RepoTags'])
  File "/root/clusterdock/clusterdock/docker_utils.py", line 81, in <genexpr>
    return any(tag == name for image in client.images() for tag in image['RepoTags'])
TypeError: 'NoneType' object is not iterable

[dimaspivak]

Thanks for reporting this, @randyzwitch! I've heard this happen to a couple of people, but can't seem to repro myself. If your machine still exhibits this, any chance you can try running the following to help me narrow down the bug?

docker run -it --entrypoint=bash -v /var/run/docker.sock:/var/run/docker.sock cloudera/clusterdock

This will open a Bash shell within the clusterdock container. At this point,
open a python interpreter by typing python and hitting enter. Then type the following:

from docker import Client
client = Client()
for image in client.images():
  print image

Does that behave? Go through without any exceptions being raised? If so, in the same interpreter, try the following:

for image in client.images():
  for tag in image['RepoTags']:
    print tag

Let me know if you can give it a shot.

[randyzwitch]

The problem is in the second snippet:

>>> for image in client.images():
...     print image['RepoTags']
... 
[u'cloudera/clusterdock:latest']
None
[u'cloudera/clusterdock:cdh580_cm581_secondary-node']
[u'cloudera/clusterdock:cdh580_cm581_primary-node']
[u'hortonworks/ambari-server:latest']
[u'hortonworks/ambari-agent:latest']
[u'nginx:latest']
[u'sequenceiq/consul:v0.5.0-v6']
[u'sequenceiq/ambari:1.7.0']
[u'sequenceiq/ambari:1.6.0']

[randyzwitch]

>>> for image in client.images():
...     for tag in image['RepoTags']:
...             print tag
... 
cloudera/clusterdock:latest
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: 'NoneType' object is not iterable

[dimaspivak]

Ah ha! Got it, looks like there are instances where the Docker API does funny things. Let me put up a fix... Thanks @randyzwitch!

[randyzwitch]

Here's this if it helps, you can see which one has the u'RepoTags': None, in it

[{u'Created': 1470342258, u'Labels': {u'org.apache.hbase.is-clusterdock': u''}, u'VirtualSize': 467454268, u'ParentId': u'', u'RepoTags': [u'cloudera/clusterdock:latest'], u'RepoDigests': [u'cloudera/clusterdock@sha256:8b125ab1c3c1edfaac9d6fd628586580bfcfa41cf72c5795151ed8e94c080bb2'], u'Id': u'sha256:f6dc042b0fb6f74d3c56febe498f857ea97298982de8da1a53f593d2675006ec', u'Size': 467454268}, {u'Created': 1470027088, u'Labels': {u'org.apache.hbase.is-clusterdock': u''}, u'VirtualSize': 467423538, u'ParentId': u'', **u'RepoTags': None**, u'RepoDigests': [u'cloudera/clusterdock@sha256:d3ed2cfd25391e052d668d56aca4f1f45ec552691669f4ea77736d541aabcab6'], u'Id': u'sha256:a4320bc89ec2028a67ba96d806dad7b5eb03a968b53b7d31e8bfbc0751fd1c46', u'Size': 467423538}, {u'Created': 1469836092, u'Labels': {}, u'VirtualSize': 4637094425, u'ParentId': u'', u'RepoTags': [u'cloudera/clusterdock:cdh580_cm581_secondary-node'], u'RepoDigests': [u'cloudera/clusterdock@sha256:6cdec6848ca6ed1d9a13174cb169762ae2103ff048d58cdc99f5d9aaecad00a6'], u'Id': u'sha256:31b6f9ea419e3670d570231ddbcb2c410bda443e89b68a01c72517f47b482818', u'Size': 4637094425}, {u'Created': 1469836026, u'Labels': {}, u'VirtualSize': 4504516048, u'ParentId': u'', u'RepoTags': [u'cloudera/clusterdock:cdh580_cm581_primary-node'], u'RepoDigests': [u'cloudera/clusterdock@sha256:396def25dbba0a9a9f371f52ed79c2266fdc451303a6bea728fe4625326b4537'], u'Id': u'sha256:1e180df693af8bf87b5b7aabb2a2ef6620d8c50943da5f97c71e51dc6aacebff', u'Size': 4504516048}, {u'Created': 1467813490, u'Labels': {u'build-date': u'2016-03-31', u'vendor': u'CentOS', u'name': u'CentOS Base Image', u'license': u'GPLv2'}, u'VirtualSize': 1613569545, u'ParentId': u'', u'RepoTags': [u'hortonworks/ambari-server:latest'], u'RepoDigests': [u'hortonworks/ambari-server@sha256:882213cc02fe18e4373fecbaba4cb1ea8c59b988dac748f3f58271cfea2fd916'], u'Id': u'sha256:9647ee697d7db319357707ddd2fcc5611382ecdbb8b84d62f370634107f0b05a', u'Size': 1613569545}, {u'Created': 1467813406, u'Labels': {u'build-date': u'2016-03-31', u'vendor': u'CentOS', u'name': u'CentOS Base Image', u'license': u'GPLv2'}, u'VirtualSize': 918029854, u'ParentId': u'', u'RepoTags': [u'hortonworks/ambari-agent:latest'], u'RepoDigests': [u'hortonworks/ambari-agent@sha256:a435f38a88ad80192fc3228333db1b75155161de2a9beaac6f88db62dc4c4caa'], u'Id': u'sha256:3ea6fc6f79cc1d32c4cdc0bc58e5ac2c4f59b9809deb98d342985a603ea8d7cd', u'Size': 918029854}, {u'Created': 1464804059, u'Labels': {}, u'VirtualSize': 182786784, u'ParentId': u'', u'RepoTags': [u'nginx:latest'], u'RepoDigests': [u'nginx@sha256:0fe6413f3e30fcc5920bc8fa769280975b10b1c26721de956e1428b9e2f29d04'], u'Id': u'sha256:0d409d33b27e47423b049f7f863faa08655a8c901749c2b25b93ca67d01a470d', u'Size': 182786784}, {u'Created': 1437568602, u'Labels': {}, u'VirtualSize': 67895602, u'ParentId': u'', u'RepoTags': [u'sequenceiq/consul:v0.5.0-v6'], u'RepoDigests': [u'sequenceiq/consul@sha256:f554045f4ce2e4422bf3f87a2362d4d908aec6952567700e10a2b324d4fa3d9d'], u'Id': u'sha256:0af2589a36013c47762115045cd4bb918712d9dafbc47ecaa7975424d4d7999d', u'Size': 67895602}, {u'Created': 1425986084, u'Labels': None, u'VirtualSize': 4761369175, u'ParentId': u'', u'RepoTags': [u'sequenceiq/ambari:1.7.0'], u'RepoDigests': [u'sequenceiq/ambari@sha256:68f148c306f37bb95e6b057b3cbfc9ab80a565d5778050c40fc51f0cbd25142c'], u'Id': u'sha256:544eb0472dd16a89bc8647766faae1ba0ea0b46ec5789f590a679091acf569f3', u'Size': 4761369175}, {u'Created': 1414862088, u'Labels': None, u'VirtualSize': 1782350976, u'ParentId': u'', u'RepoTags': [u'sequenceiq/ambari:1.6.0'], u'RepoDigests': [u'sequenceiq/ambari@sha256:8b2daa7997cf9758da3aac7348ae10e2abf35b6a39b226ce6ad311a2b0bfad0f'], u'Id': u'sha256:2078c7eddcbf7565b509049bd4e9f49a577c28a8cc78abf7c912b14c5b4f750c', u'Size': 1782350976}]

[dimaspivak]

Just pushed an updated image, @randyzwitch. Can you give it a try to see if it fixed the issue? (Just rerun the command as before. The clusterdock helper script automatically pulls down new images.)

[randyzwitch]

Yes, that fixed it, thanks!