Expected relations_ to be a Sequence, received: <class 'dict'>

[lourot]

With the latest release of libjuju (2.8.0) we (OpenStack) are hitting this error in all our Zaza tests:

2020-05-13 09:24:57 [INFO] ======================================================================
2020-05-13 09:24:57 [INFO] ERROR: setUpClass (zaza.openstack.charm_tests.barbican_vault.tests.BarbicanVaultUnsealVault)
2020-05-13 09:24:57 [INFO] ----------------------------------------------------------------------
2020-05-13 09:24:57 [INFO] Traceback (most recent call last):
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/openstack/charm_tests/barbican_vault/tests.py", line 27, in setUpClass
2020-05-13 09:24:57 [INFO]     super(BarbicanVaultUnsealVault, cls).setUpClass()
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/openstack/charm_tests/vault/tests.py", line 98, in setUpClass
2020-05-13 09:24:57 [INFO]     super(UnsealVault, cls).setUpClass()
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/openstack/charm_tests/vault/tests.py", line 44, in setUpClass
2020-05-13 09:24:57 [INFO]     "vault", model_name=cls.model_name)
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/__init__.py", line 48, in _wrapper
2020-05-13 09:24:57 [INFO]     return run(_run_it())
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/__init__.py", line 36, in run
2020-05-13 09:24:57 [INFO]     return task.result()
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/__init__.py", line 47, in _run_it
2020-05-13 09:24:57 [INFO]     return await f(*args, **kwargs)
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/zaza/model.py", line 482, in async_get_lead_unit_name
2020-05-13 09:24:57 [INFO]     is_leader = await unit.is_leader_from_status()
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/unit.py", line 264, in is_leader_from_status
2020-05-13 09:24:57 [INFO]     status = await client_facade.FullStatus(patterns=None)
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/client/facade.py", line 504, in wrapper
2020-05-13 09:24:57 [INFO]     result = cls.from_json(reply['response'])
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/client/facade.py", line 639, in from_json
2020-05-13 09:24:57 [INFO]     return cls(**d)
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/client/_definitions.py", line 9101, in __init__
2020-05-13 09:24:57 [INFO]     applications_ = {k: ApplicationStatus.from_json(v) for k, v in (applications or dict()).items()}
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/client/_definitions.py", line 9101, in <dictcomp>
2020-05-13 09:24:57 [INFO]     applications_ = {k: ApplicationStatus.from_json(v) for k, v in (applications or dict()).items()}
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/client/facade.py", line 639, in from_json
2020-05-13 09:24:57 [INFO]     return cls(**d)
2020-05-13 09:24:57 [INFO]   File "/home/ubuntu/barbican-vault/charm-barbican-vault/src/.tox/func-smoke/lib/python3.6/site-packages/juju/client/_definitions.py", line 2716, in __init__
2020-05-13 09:24:57 [INFO]     raise Exception("Expected relations_ to be a Sequence, received: {}".format(type(relations_)))
2020-05-13 09:24:57 [INFO] Exception: Expected relations_ to be a Sequence, received: <class 'dict'>
2020-05-13 09:24:57 [INFO] ----------------------------------------------------------------------
2020-05-13 09:24:57 [INFO] Ran 0 tests in 0.316s
2020-05-13 09:24:57 [INFO] FAILED
2020-05-13 09:24:57 [INFO]  (errors=1)

This is hitting us both with juju 2.7.6 and 2.8-rc1. Going back to libjuju 2.7.1 seems to solve the issue.

We suspect #412 broke it.

[SimonRichardson]

From the looks of it, either the facade parameters changed or the python generated client code is broken. I suspect it's the former, without fully looking.

(Currently on holiday, back Monday if it's not resolved by then)

[pengale]

So from an initial look at this, the problem might be this block of code in juju/client/_definitions.py (line 9101):

        applications_ = {k: ApplicationStatus.from_json(v) for k, v in (applications or dict()).items()}
        branches_ = {k: BranchStatus.from_json(v) for k, v in (branches or dict()).items()}
        controller_timestamp_ = controller_timestamp
        machines_ = {k: MachineStatus.from_json(v) for k, v in (machines or dict()).items()}
        model_ = ModelStatusInfo.from_json(model) if model else None
        offers_ = {k: ApplicationOfferStatus.from_json(v) for k, v in (offers or dict()).items()}
        relations_ = [RelationStatus.from_json(o) for o in relations or []]
        remote_applications_ = {k: RemoteApplicationStatus.from_json(v) for k, v in (remote_applications or dict()).items()}

Note that we are defaulting, when something doesn't exist, to a dict, which might be failing the type check from the latest raft of definitions.

This is just an educated guess at this point -- I'm coming back to pylibjuju after a long absence. I'm not sure whether the "right" fix is to introspect and figure out the correct defaults, or to be more forgiving about what we get passed in the schema ...

[SimonRichardson]

@petevg if it's the latter, then upstream juju broke the facade parameters and needs fixing there.

[hpidcock]

@AurelienLourot can you please test with https://github.com/juju/python-libjuju/tarball/master#egg=juju-2.8.0

[dshcherb]

@hpidcock

Ran into the following with 9e1409a:

2020-05-14 13:46:21 [INFO] ERROR                                                                                                                                      
2020-05-14 13:46:21 [INFO] ======================================================================                                                                     
2020-05-14 13:46:21 [INFO] ERROR: setUpClass (zaza.openstack.charm_tests.designate.tests.DesignateTests)                                                              
2020-05-14 13:46:21 [INFO] ----------------------------------------------------------------------                                                                     
2020-05-14 13:46:21 [INFO] Traceback (most recent call last):                                                                                                         
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/openstack/charm_tests/designate/tests.py", line 35
, in setUpClass                                                                                                                                                       
2020-05-14 13:46:21 [INFO]     super(BaseDesignateTest, cls).setUpClass(application_name, model_alias)                                                                
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/openstack/charm_tests/test_utils.py", line 430, in
 setUpClass                                                                                                                                                           
2020-05-14 13:46:21 [INFO]     super(OpenStackBaseTest, cls).setUpClass(application_name, model_alias)
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/openstack/charm_tests/test_utils.py", line 128, in
 setUpClass                                                                                                                                                           
2020-05-14 13:46:21 [INFO]     cls.lead_unit = model.get_lead_unit_name(                                                                                              
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/__init__.py", line 48, in _wrapper
2020-05-14 13:46:21 [INFO]     return run(_run_it())                                                                                                                  
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/__init__.py", line 36, in run
2020-05-14 13:46:21 [INFO]     return task.result()                                                                                                                   
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/__init__.py", line 47, in _run_it
2020-05-14 13:46:21 [INFO]     return await f(*args, **kwargs)                                                                                                        
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/zaza/model.py", line 482, in async_get_lead_unit_name
2020-05-14 13:46:21 [INFO]     is_leader = await unit.is_leader_from_status()                                                                                         
2020-05-14 13:46:21 [INFO]   File "/home/ubuntu/charm-designate-bind/src/.tox/func/lib/python3.8/site-packages/juju/unit.py", line 282, in is_leader_from_status
2020-05-14 13:46:21 [INFO]     if status.applications[app].get('subordinate-to'):                                                                                     
2020-05-14 13:46:21 [INFO] AttributeError: 'ApplicationStatus' object has no attribute 'get'      
2020-05-14 13:46:21 [INFO] ----------------------------------------------------------------------

[hpidcock]

Fixed in #415 and #416