Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up the processing of machines in juju status. #11310

Merged
merged 1 commit into from Mar 12, 2020

Conversation

@howbazaar
Copy link
Member

howbazaar commented Mar 12, 2020

This branch addresses the multiple database queries that were being
run for every machine in the generation of the full status results.

The constraints were being read individually, so we read them in bulk
at the start of the function. State tests were augmented to show that
constraints set on a machine can be read from all the constraints.

Additionally the instance data was being queries three times for
every machine. Once to get the instance id and display name,
another time for the hardware characteristics, and a third time
for the charm profiles. The instance data is now read in bulk at
the start of status and used when processing the machines.

Without these changes the test results showed:

   c.Check(tracker.ReadCount(), gc.Equals, queryCount,
        gc.Commentf("if the query count is not the same, there has been a regression "+
            "in the processing of machines, please fix it"))
... obtained int = 41
... expected int = 21

Which matches expectations, as five machines were added, and the
processing of each added another four queries.

Bug reference

https://bugs.launchpad.net/juju/+bug/1865172

This branch addresses the multiple database queries that were being
run for every machine in the generation of the full status results.

The constraints were being read individually, so we read them in bulk
at the start of the function. State tests were augmented to show that
constraints set on a machine can be read from all the constraints.

Additionally the instance data was being queries three times for
every machine. Once to get the instance id and display name,
another time for the hardware characteristics, and a third time
for the charm profiles. The instance data is now read in bulk at
the start of status and used when processing the machines.

Without these changes the test results showed:
   c.Check(tracker.ReadCount(), gc.Equals, queryCount,
        gc.Commentf("if the query count is not the same, there has been a regression "+
            "in the processing of machines, please fix it"))
... obtained int = 41
... expected int = 21

Which matches expectations, as five machines were added, and the
processing of each added another four queries.
@tlm
tlm approved these changes Mar 12, 2020
Copy link
Member

tlm left a comment

lgtm

@howbazaar

This comment has been minimized.

Copy link
Member Author

howbazaar commented Mar 12, 2020

Check job has intermittent failures...

OOPS: 13 passed, 4 FAILED
--- FAIL: TestPackage (66.63s)
FAIL
FAIL	github.com/juju/juju/worker/caasoperator	81.934s

$$merge$$

@jujubot jujubot merged commit b3dff5d into juju:2.7 Mar 12, 2020
3 of 5 checks passed
3 of 5 checks passed
Client Tests (ubuntu-latest) Client Tests (ubuntu-latest)
Details
Lint
Details
Schema
Details
check-multi-juju Build finished.
Details
merge-multi-juju Build started for merge commit.
Details
@howbazaar howbazaar deleted the howbazaar:2.7-faster-machine-status branch Mar 12, 2020
jujubot added a commit that referenced this pull request Mar 13, 2020
#11314

## Description of change

Merge 2.7 bringing in these PRs:

#11286 speed up juju status by handling units better
#11296 fix LP:1860083 machine availability zone
#11291 various refactorings (method renames, tools finder/getter handle k8s broker)
#11297 juju show-action-status not listing all
#11300 k8s controllers can start lxd containers
#11304 protect against nil profile from the charm
#11310 speed up processing of machines in juju status
#11306 ensure k8s juju upgrades select correct agent version
#11313 fix watcher race in ControllerAddressSuite
#11302 fix LP:1866658 jujud called in makefile before it is built

## QA steps

Run unit tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.