Glitch should only apply changes to apps deployed by the test

[johnsca]

While we should generally run matrix tests on clean models, for dev / testing it's often helpful to (re)run against a pre-deployed model. However, if the current model includes applications that are not part of the matrix test, we might cause mutations to them which are not relevant to the test.

I think glitch should start from context.apps instead of context.juju_model.applications.

[petevg]

I've actually been taking advantage of the fact that matrix will go to town on stuff by deploying things by hand in advance, and then running just the glitch bits of matrix.

That was largely a workaround for when the deploy bits of matrix weren't stable, though. I suppose that need will go away with the actual production version of matrix.

[johnsca]

I should clarify; as long as the apps are mentioned in the bundle, they will be added to context.apps, even if they were pre-deployed. This should already work, even with the issues with libjuju.

My concern is that the tests will be run with a specific set of charms in mind, and if there are other, unrelated apps in the environment, we should ignore them. Currently, the health check only checks apps in the context.apps list. I suppose another option would be to make all tasks look at the model as a whole, but that seems a bit messier.

[petevg]

@johnsca Aha. Got it. That makes a lot of sense.

[petevg]

@johnsca Is this addressed to your satisfaction by #43?

[johnsca]

Yep.