Support a simplifed dump-model #7471

Merged
merged 3 commits into from Jun 8, 2017

Conversation

Projects
None yet
3 participants
Owner

howbazaar commented Jun 8, 2017

Description of change

juju status is shown to be terribly slow on very large systems. This is mostly due to how complexity of status has grown over time and the O(n²) operations it has. This code adds an arg struct to state.Export to allow a simplified export which could be used in the status generation. This allows us to test the timing of the information gathering aspect of what status could be in order to determine if the work is worthwhile doing.

QA steps

juju bootstrap lxd testing
juju deploy ubuntu
JUJU_DEV_FEATURE_FLAGS=developer-mode juju dump-model

observe that status history, ssh host keys, model config is in the output

JUJU_DEV_FEATURE_FLAGS=developer-mode juju dump-model --simplified

observe that those values are empty.

Documentation changes

None at this stage.

Owner

howbazaar commented Jun 8, 2017

The output format has changed to deal with a bug in dump-model where all the integers in the model description came out in the YAML as floats.

This was because the model was deserialized in the apiserver into map[string]interface{} and then passed over the JSON RPC methods, which only knows floats. When the map on the other side was converted to YAML, the numbers stayed floats.

Some minor things, including a missing test.
If you use the facade context form of the api facade New() method, you get the state pool right there and it's trivial to then use it instead of ForModel()

apiserver/modelmanager/modelmanager.go
@@ -353,6 +382,7 @@ func (m *ModelManagerAPI) dumpModel(args params.Entity) (map[string]interface{},
st := m.state
if st.ModelTag() != modelTag {
+ // XXX state pool
@wallyworld

wallyworld Jun 8, 2017

Owner

It's stuff all work to use the state pool - could we do it as a drive by?

@howbazaar

howbazaar Jun 8, 2017

Owner

Yes, I intended to. Done now.

apiserver/modelmanager/modelmanager.go
@@ -412,10 +467,26 @@ func (m *ModelManagerAPI) dumpModelDB(args params.Entity) (map[string]interface{
return st.DumpAll()
}
+func (m *ModelManagerAPI) DumpModels(args params.DumpModelRequest) params.StringResults {
@wallyworld

wallyworld Jun 8, 2017

Owner

Needs a comment similar to the V2 method below which does have a comment

@@ -54,6 +55,7 @@ func (c *dumpCommand) Info() *cmd.Info {
func (c *dumpCommand) SetFlags(f *gnuflag.FlagSet) {
c.ControllerCommandBase.SetFlags(f)
c.out.AddFlags(f, "yaml", output.DefaultFormatters)
+ f.BoolVar(&c.simplified, "simplified", false, "Dump a simplified partial model")
@wallyworld

wallyworld Jun 8, 2017

Owner

Good that this defaults to false, but we should print a warning if true and only v2 api is available, lest the user expects they will get a simplified model and then they don't get one.

@howbazaar

howbazaar Jun 8, 2017

Owner

Good point.

@@ -32,14 +32,15 @@ func (f *fakeDumpClient) Close() error {
return f.NextErr()
}
-func (f *fakeDumpClient) DumpModel(model names.ModelTag) (map[string]interface{}, error) {
+func (f *fakeDumpClient) DumpModel(model names.ModelTag, simplified bool) (map[string]interface{}, error) {
@wallyworld

wallyworld Jun 8, 2017

Owner

We should record the value of simplified and in TestDumpSimple verify that it was indeed as expected

@howbazaar

howbazaar Jun 8, 2017

Owner

I encode it in the response map and assert in the tests.

+
+// ExportPartial the current model for the State optionally skipping
+// aspects as defined by the ExportConfig.
+func (st *State) ExportPartial(cfg ExportConfig) (description.Model, error) {
@wallyworld

wallyworld Jun 8, 2017

Owner

I can't see any tests for this?

@howbazaar

howbazaar Jun 8, 2017

Owner

added some tests

Owner

howbazaar commented Jun 8, 2017

$$merge$$

Contributor

jujubot commented Jun 8, 2017

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

@jujubot jujubot merged commit 2ff4965 into juju:2.2 Jun 8, 2017

1 check failed

github-check-merge-juju Ran tests against PR. Use !!.*!! to request another build. IE, !!build!!, !!retry!!
Details

@howbazaar howbazaar deleted the howbazaar:2.2-state-export-tweaks branch Jun 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment