/juju

Transaction pruning improvments #7183

Merged
merged 2 commits into from Mar 31, 2017

Conversation

Reviewers

@jameinel jameinel

@axw axw

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@mjs @jujubot @jameinel @axw
@mjs
Contributor

mjs commented Mar 31, 2017

Description of change

Two parts:

  1. Update to newest juju/txn to pull in the improved txn pruning algorithm.

  2. Prune transactions after 10% growth. Previously we would only prune after 2x growth which wasn't aggressive enough and made the prune process impact system resources much more
    when it did run. Now that the pruning process is much more efficient running it more often is ok.

QA steps

  • Bootstrap a system with the txnpruner worker hacked to run every 20s instead of every 2 hours.
  • Generate transactions by deploying and relating applications
  • Observed that the pruner ran when it was supposed and did the right things:
$ juju debug-log -m controller --include-module juju.txn

Documentation changes

N.A. (internal only)

Bug reference

https://bugs.launchpad.net/juju/+bug/1676427

mjs added some commits Mar 31, 2017

@mjs
Update to newest juju/txn 
This pulls in the improved txn pruning algorithm.

Part of the fix for https://bugs.launchpad.net/juju/+bug/1676427
4b919df
@mjs
state: Prune transactions after 10% growth since last prune 
Previously we would only prune after 2x growth which wasn't aggressive
enough and made the prune process impact system resources much more
when it did run. Now that the pruning process is much more efficient
running it more often is ok.
819810f
@axw

axw approved these changes Mar 31, 2017
View changes

@mjs
Contributor

mjs commented Mar 31, 2017

$$merge$$
@jujubot
Contributor

jujubot commented Mar 31, 2017

Status: merge request accepted. Url: http://juju-ci.vapour.ws:8080/job/github-merge-juju

@jujubot jujubot merged commit 42c12c8 into juju:develop Mar 31, 2017

@mjs mjs deleted the mjs:1676427-txn-pruning branch Mar 31, 2017

state/txns.go
- // Prune txns only when txn count has doubled since last prune.
- return runner.MaybePruneTransactions(2.0)
+ // Prune txns when txn count has increased by 10% since last prune.
+ return runner.MaybePruneTransactions(1.1)
@jameinel

jameinel Mar 31, 2017

Owner

I do wonder if we could have some sort of an absolute threshold as well. "Prune if transactions have grown by 1M regardless of the % increase".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment