Need more control over pre-test deployment of bundle

[johnsca]

When testing a bundle, BT deploys the bundle, then runs the charm tests, before finally running the bundle tests. This doesn't give the bundle test a chance to affect the deployment, meaning the only option in some cases is to reset the env and redeploy, which can make long tests take longer.

Suggested additions to test.yaml options:

• pre_deploy (default: true) Allows you to disable the automatic pre-deployment of the bundle
• charm_tests_first (default: true) Allows you to have the bundle tests run before the charm tests

If both of these are changed, it would mean that the bundle tests are responsible for deploying the bundle and have full control over that deployment. The charm tests could still re-use that environment, assuming reset is disabled and the service names match.

[johnsca]

Perhaps another possibility would be to add an option that lets the bundle specify a test case (e.g., 00-setup) that replaces the automatic pre-deployment step (with the order otherwise being unaffected)?

[tvansteenburgh]

So we could simplify this down to:

bundle_deploy: auto | <test name>

@johnsca does that work for you?

[tvansteenburgh]

Or maybe:

bundle_deploy: true | false | <test name>

@johnsca Thoughts?

[johnsca]

Nice and clean, accomplishes what we need. +1