Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/jujud: synchronise state server upgrades #709

Merged
merged 3 commits into from Sep 9, 2014
Merged

Conversation

mjs
Copy link

@mjs mjs commented Sep 8, 2014

Use state's UpgradeInfo to synchronise state server upgrades. Overview:

  • after restarting into a new tools version, master state servers will wait up to 15 minutes for any other state servers to indicate they are ready for upgrade. A downgrade to the previous tools version will occur if this wait times out.
  • secondary state servers will wait until the master has completed its upgrade (to allow DB schema migrations to happen first)
  • as state servers complete their upgrades they report this to the UpgradeInfo document. The document is archived once the last state server completes its upgrade.

Note that some refactoring to the implementation and tests is planned immediately after this PR lands. I didn't include them here because I didn't want to clutter up this PR any further.

In preparation for upcoming upgrade changes:
- machines created by primeAgent now have a pinger running so that
  EnsureAvailability can be used in tests to create additional state
  servers
- separated machine configuration from creation in primeAgent to allow
  machines created through other means (e.g. EnsureAvailability) to
  still be configured for use with machine agents
@@ -155,16 +184,24 @@ func (c *upgradeWorkerContext) runUpgrades() error {

a := c.agent
tag := c.agentConfig.Tag().(names.MachineTag)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line just gives me the heebie jeebies. Mostly because it panics if the tag isn't a machine tag.
I know it was there before, but we really prefer no panic possibilities in production code.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could argue that if a machine agent's tag isn't a MachineTag then things are so bad that panicking might be the best course of action :)

Still, I will change this.

@howbazaar
Copy link
Contributor

looks pretty good, just a few questions

Use state's UpgradeInfo to synchronise state server upgrades. Overview:

- after restarting into a new tools version, master state servers will
  wait up to 15 minutes for any other state servers to indicate they
  are ready for upgrade. A downgrade to the previous tools version
  will occur if this wait times out.
- secondary state servers will wait until the master has completed its
  upgrade (to allow DB schema migrations to happen first)
- as state servers complete their upgrades they report this to the
  UpgradeInfo document. The document is archived once the last state
  server completes its upgrade.
@howbazaar
Copy link
Contributor

LGTM

@mjs
Copy link
Author

mjs commented Sep 9, 2014

$$merge$$

@jujubot
Copy link
Collaborator

jujubot commented Sep 9, 2014

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

jujubot added a commit that referenced this pull request Sep 9, 2014
cmd/jujud: synchronise state server upgrades

Use state's UpgradeInfo to synchronise state server upgrades. Overview:
    
- after restarting into a new tools version, master state servers will wait up to 15 minutes for any other state servers to indicate they are ready for upgrade. A downgrade to the previous tools version will occur if this wait times out.
- secondary state servers will wait until the master has completed its upgrade (to allow DB schema migrations to happen first)
- as state servers complete their upgrades they report this to the UpgradeInfo document. The document is archived once the last state server completes its upgrade.

Note that some refactoring to the implementation and tests is planned immediately after this PR lands. I didn't include them here because I didn't want to clutter up this PR any further.
@jujubot jujubot merged commit 768fcf7 into juju:master Sep 9, 2014
@mjs mjs deleted the upgrade-sync branch September 9, 2014 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants