Use sequences to track used charm URLs #6866

Merged
merged 8 commits into from Jan 30, 2017

Conversation

Projects
None yet
4 participants
Contributor

mjs commented Jan 25, 2017

Instead of relying on "dead" charm documents, which doesn't work well with migrations, local charm revisions are now tracked using a sequence. Dead charm documents are now removed.

An upgrade step is also included which adds any required sequences for pre-existing charms.

This PR also includes a fix for an unrelated gofmt issue in worker/machiner.

QA steps

  • Deploy using 2.1 branch
  • Deploy a local charm
  • Remove the local charm
  • juju upgrade-juju to the code in this PR.
  • Observe the the DB now contains the appropriate sequence & that the dead charm doc for the previous deploy is removed.
  • Redeploy the local charm, observing the next charm revision is used.
  • Deploy another application using the same local charm URL.
  • Remove both applications, observing the charm is removed from the charms collection and the blobstore only once both applications are gone.
  • Redeploy the local charm and observe that correct sequence is used.

Bug reference

This is the final fix for https://bugs.launchpad.net/juju/+bug/1657614

@@ -183,3 +189,25 @@ func (su *dbSeqUpdater) set(expected, next int) (bool, error) {
}
return true, nil
}
+
+func (su *dbSeqUpdater) ensure(next int) error {
@wallyworld

wallyworld Jan 25, 2017

Owner

there should probably be a unit test for this new function

@mjs

mjs Jan 26, 2017

Contributor

Done

state/sequence.go
+ ok, err = su.set(curVal, next)
+ }
+ if !ok {
+ return errors.New("unexpected contention")
@wallyworld

wallyworld Jan 25, 2017

Owner

Elsewhere it says "too much contention while updating sequence"
Can we make this a little verboser too?

@mjs

mjs Jan 26, 2017

Contributor

Done

@@ -57,13 +56,9 @@ func (st *State) sequence(name string) (int, error) {
// `sequence` is more efficient than `sequenceWithMin` and should be
// preferred if there is no minimum value requirement.
func (st *State) sequenceWithMin(name string, minVal int) (int, error) {
- sequences, closer := st.getCollection(sequenceC)
+ sequences, closer := st.getRawCollection(sequenceC)
@wallyworld

wallyworld Jan 25, 2017

Owner

This change implies a gap in unit test coverage?
Do there need to be multi-model tests added?

@mjs

mjs Jan 25, 2017

Contributor

The existing tests already check multi-model behaviour. This is an internal only change.

- // Note that this aligns with the existing contract implied by
- // Dead: that most clients should see it as not existing at all.
- // Nothing strictly obliges us to clean up the doc.
+ // immortal.
@wallyworld

wallyworld Jan 25, 2017

Owner

There should be changes to the charm unit tests to ensure that removing the dead docs still sees the correct revision being used.

@mjs

mjs Jan 25, 2017

Contributor

There are preexisting tests which check this

@@ -38,12 +38,12 @@ func (s *metricsDebugSuite) SetUpTest(c *gc.C) {
}
func (s *metricsDebugSuite) TestSetMeterStatus(c *gc.C) {
- testCharm := s.Factory.MakeCharm(c, &factory.CharmParams{Name: "metered", URL: "local:quantal/metered"})
+ testCharm := s.Factory.MakeCharm(c, &factory.CharmParams{Name: "metered", URL: "local:quantal/metered-1"})
@wallyworld

wallyworld Jan 25, 2017

Owner

There are other places that may need the same change, eg apiserver/uniter_test

@mjs

mjs Jan 25, 2017

Contributor

I ran all the tests under apiserver/...

axw approved these changes Jan 25, 2017

Overall LGTM.

Contributor

mjs commented Jan 26, 2017

$$merge$$

Contributor

jujubot commented Jan 26, 2017

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

Contributor

jujubot commented Jan 26, 2017

Build failed: Tests failed
build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/10117

Contributor

mjs commented Jan 29, 2017

$$test-infra-error$$

Contributor

jujubot commented Jan 29, 2017

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

Contributor

jujubot commented Jan 29, 2017

Build failed: Tests failed
build url: http://juju-ci.vapour.ws:8080/job/github-merge-juju/10131

mjs added some commits Jan 24, 2017

Fix metrics tests which use invalid charm URLs
Various metrics tests were using charm URLs without a revision. This
shouldn't have been allowed and will no longer be possible with upcoming
changes.
state: Track charm URLs using a sequence
Instead of relying on dead charm documents, use a sequence to track
used charm URL revisions.
state: Added dbSeqUpdater.ensure
This will be used for an upcoming upgrade step.

Also now uses a raw collection when updating sequences, also to simplify
upgrade steps.
Upgrade step to add local charm revision sequences
Local charm revisions that have been used are now tracked with a
sequence. An upgrade step has been added which ensures that these
sequences exist for preexisting charms.
state: Remove charm docs once unused
Instead of leaving dead charm documents around they are now removed once
no longer required.
state: Remove dead charm doc on upgrade
The upgrade step which creates sequences to track local charm revisions
now ends by removing dead charm documents. These are no longer required.
state: Add tests for dbSeqUpdater.ensure
These were missed when this function was added.
Contributor

mjs commented Jan 30, 2017

$$merge$$

Contributor

jujubot commented Jan 30, 2017

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

@jujubot jujubot merged commit 7048e51 into juju:2.1 Jan 30, 2017

@mjs mjs deleted the mjs:change-charm-url-tracking branch Jan 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment