DeadEntity in Machine._workaround

[johnsca]

DeadEntityException: Entity machine:2 is dead - its attributes can no longer be accessed. Use the .previous() method on this object to get a copy of the object at its previous state.
  File "juju/model.py", line 57, in __call__
    await self.callable_(delta, old, new, model)
  File "juju/machine.py", line 46, in _workaround_1695335
    if self.id not in full_status.machines:
  File "juju/model.py", line 243, in __getattr__
    return self.safe_data[name]
  File "juju/model.py", line 329, in safe_data
    self.entity_type, self.entity_id))

Seems like an issue when a machine is being removed. Originally from sentry: https://sentry.io/canonical-pj/conjure-up/issues/302200453/

[johnsca]

I wasn't able to reproduce this, and it happens pretty infrequently in Sentry, so it seems like it might be a race condition. I'm also not sure why a machine would be dead in a conjure-up deploy unless someone did something manually from outside of conjure-up.

[matuskosut]

I am now working with very active dev-stack consisting of openstack applications where I execute a lot of juju actions on all the units. What I observe is excessively growing delta due to storing results of all actions until the model is deleted. All of this is sent over rpc in one message, considering the very low default limit of 4MB (MAX_FRAME_SIZE = 2**22) I believe the update of delta in client fails silently, which results in having inconsistent information about machines and probably other issues.

Just to compare my frame size now is 64MB. Hitting this results in inability to connect until the limit is increased, which is why I started to look into it and actually realised I have seen this same error when cleaning up machines from my model. So I will have to try to reproduce soon to catch this one.

[juanmanuel-tirado]

This issue has expired.