state: delete status history in batches #8254
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When removing status history docs because a unit was destroyed, remove the docs in batches to avoid locking the collection for a long period of time.
howbazaar
reviewed
Jan 3, 2018
| lastUpdate = now | ||
| } | ||
| } | ||
| } | ||
| if err := iter.Close(); err != nil { |
There was a problem hiding this comment.
Now we are closing the iterator both here and in the calling function?
I guess this is fine as long as calling close twice is fine.
There was a problem hiding this comment.
Close is documented as being idempotent.
| @@ -208,16 +217,19 @@ func (p *collectionPruner) deleteInBatches(iter *mgo.Iter, logTemplate string, s | |||
|
|
|||
| now := time.Now() | |||
| if now.Sub(lastUpdate) >= historyPruneProgressSeconds*time.Second { | |||
There was a problem hiding this comment.
Do we care about the historyPruneProgressSeconds value?
There was a problem hiding this comment.
I would say that if the deletion takes more than 15s in total, yes we do.
1000 documents. |
howbazaar
approved these changes
Jan 4, 2018
|
|
|
Status: merge request accepted. Url: http://ci.jujucharms.com/job/github-merge-juju |
jujubot
added a commit
that referenced
this pull request
Jan 5, 2018
Backport status history deletion and mongo iterator closure to 2.2 ## Description of change Backport #8254 and #8255 to the 2.2 branch. ## QA steps Smoke test bootstrap/deploy/destroy-controller. ## Documentation changes None. ## Bug reference https://bugs.launchpad.net/juju/+bug/1733708 https://bugs.launchpad.net/juju/+bug/1739380
jameinel
reviewed
Jan 7, 2018
| iter := history.Find(bson.D{{ | ||
| globalKeyField, globalKey, | ||
| }}).Select(bson.M{"_id": 1}).Iter() | ||
| defer iter.Close() |
There was a problem hiding this comment.
Any chance we could update this to at least log errors from Close()?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
When removing status history docs because
a unit was destroyed, remove the docs in
batches to avoid locking the collection
for a long period of time.
QA steps
smoke test bootstrap / deploy / remove-application
(check status history docs removed)
Also, I ran a before/after experiment:
After the change, takes ~10s less time overall (~80s -> ~70s)
Documentation changes
None.
Bug reference
May help with https://bugs.launchpad.net/juju/+bug/1733708
Also partially addresses https://bugs.launchpad.net/juju/+bug/1739380 (closes iterators in pruning code)