Caching store tracker #157
Conversation
…n we are going over maximum size
…aching-store-imp
…etStore is lossy, since it only has 5 possible places to put each key, so change howManyBlocks in howManyBlocks*5
… logic in pushAll
…lush job that gets called when it's full
…aching-store-tracker
…/unregister and take snapshot in pushAllCachingStores
…rCachingFS in start
…ushed down to the *underlying* store
…rCachingFS, improvements and fixing in add
…ExpireSSK and waiting the flushing of the cache in testOverMaximumSize
…ngStores.size()]) to avoid casting from Object, delete pushAllCachingStores call direclty in add
…hAllCachingStores
…store-tracker Conflicts: src/freenet/node/Node.java
…re), check for size in pushAll loop without having to lock again.
…shAllCachingStores.
…e. queuedJob prevents us from queueing two or more jobs to write everything in the future.
|
Build results will soon be (or already are) available at: https://jenkins.freenetproject.org/job/freenet-pullreq/33/ |
|
Build results will soon be (or already are) available at: https://jenkins.freenetproject.org/job/freenet-pullreq/35/ |
|
Build results will soon be (or already are) available at: https://jenkins.freenetproject.org/job/freenet-pullreq/36/ |
|
Build results will soon be (or already are) available at: https://jenkins.freenetproject.org/job/freenet-pullreq/48/ |
|
retest this please |
|
Build results will soon be (or already are) available at: https://jenkins.freenetproject.org/job/freenet-pullreq/58/ |
|
What does this do? |
|
Locking enhancements related to lazy writing in the datastore IIRC. There are a couple of bugs for it on the bug tracker... |
|
Doh; I didn't realize this was pending on me... I've had a look. CachingFreenetStoreTracker.unregisterCachingFS() is broken locking wise; but imho it doesn't matter in real-world usage as we're not unregistering stuff except when we shutdown... The simplest way to fix it is to ensure that we have the write lock in CachingFreenetStore.close() when we're calling it. @voxsim can you update the pull request with that single change? The rest looks good to field-test/merge to me; the other two bugs pointed out by toad are further optimizations. |
|
Is this still current with purge-db4o? |
|
purge-db4o is irrelevant to this commit. The original pull request was merged, this is optimisations / locking changes. |
|
any updates on this? |
|
Not that I know of, are you volunteering to update it with the locking change I've suggested in #157 (comment) ? I'd be great if that went in |
This reverts commit c944f1f.
|
Ok; so that's done. I suggest we wait for a release with no other major change before it makes it to a released build though... |
|
wow - thanks a lot! Last week I wanted to ask whether this would really just entail adding another lock-section around the unregister - now I see it’s even easier… |
|
This is risky, are we fairly sure about it? What testing can we do on it? |
|
The easiest test is a testing release after the big releases are done. |
|
Well if it's been carefully reviewed then go ahead, after thorough prerelease testing. It's just one of the few changes that could actually cause e.g. deadlocks, inability to update. |
|
I plan to merge this for 1469. |
|
bump |
|
|
People have been running the prereleases, yes. For finding deadlocks review On Tue, Dec 8, 2015, 10:30 PM xor-freenet notifications@github.com wrote:
|
No description provided.