Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add facility to retrieve and keep data hot and locked #1254

Merged
merged 20 commits into from Jun 5, 2019
Merged

Add facility to retrieve and keep data hot and locked #1254

merged 20 commits into from Jun 5, 2019

Conversation

jnwatson
Copy link
Contributor

This will prefault and lock all the memory for a particular lmdbslab.
Add a configuration option to the cortex and lmdblayer to enable this.

@vEpiphyte vEpiphyte added this to the v0.1.10 milestone May 31, 2019
rakuy0
rakuy0 reviewed May 31, 2019
View reviewed changes
Copy link
Contributor

@rakuy0 rakuy0 left a comment •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few nitpicks here and there, but otherwise looks reasonable and lines up with all our the earlier talks.

synapse/cortex.py Outdated Show resolved Hide resolved
synapse/lib/platforms/linux.py Show resolved Hide resolved
synapse/lib/lmdbslab.py Outdated Show resolved Hide resolved
synapse/cortex.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented May 31, 2019

Codecov Report

Merging #1254 into master will decrease coverage by 18.09%.
The diff coverage is 16.02%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1254      +/-   ##
==========================================
- Coverage   94.12%   76.02%   -18.1%     
==========================================
  Files         126      126              
  Lines       16416    16561     +145     
==========================================
- Hits        15451    12591    -2860     
- Misses        965     3970    +3005
Impacted Files Coverage Δ
synapse/lib/platforms/common.py 46.42% <0%> (ø) ⬆️
synapse/lib/platforms/linux.py 0% <0%> (-100%) ⬇️
synapse/lib/lmdbslab.py 65.19% <25%> (-33.43%) ⬇️
synapse/lib/lmdblayer.py 82.5% <80%> (-6.79%) ⬇️
synapse/cortex.py 86.47% <83.33%> (-5.01%) ⬇️
synapse/tools/guid.py 0% <0%> (-100%) ⬇️
synapse/lib/crypto/tinfoil.py 0% <0%> (-100%) ⬇️
synapse/tools/easycert.py 0% <0%> (-100%) ⬇️
synapse/lib/storm_format.py 0% <0%> (-100%) ⬇️
synapse/servers/cell.py 0% <0%> (-100%) ⬇️
... and 58 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 725e743...d94640d. Read the comment docs.

@codecov
Copy link

codecov bot commented May 31, 2019 •
edited

Codecov Report

Merging #1254 into master will increase coverage by 0.05%.
The diff coverage is 97.04%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1254      +/-   ##
==========================================
+ Coverage   94.16%   94.21%   +0.05%     
==========================================
  Files         126      126              
  Lines       16457    16622     +165     
==========================================
+ Hits        15497    15661     +164     
- Misses        960      961       +1
Impacted Files Coverage Δ
synapse/lib/lmdblayer.py 89.4% <100%> (+0.1%) ⬆️
synapse/lib/lmdbslab.py 98.83% <100%> (+0.21%) ⬆️
synapse/lib/base.py 88.7% <100%> (+1.03%) ⬆️
synapse/cortex.py 92.22% <100%> (+0.46%) ⬆️
synapse/lib/platforms/common.py 50% <100%> (+3.57%) ⬆️
synapse/lib/platforms/linux.py 94.04% <93.75%> (-5.96%) ⬇️
synapse/lib/link.py 91.96% <0%> (-4.47%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d81127...b0595d5. Read the comment docs.

vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/cortex.py Outdated Show resolved Hide resolved
vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/cortex.py Outdated Show resolved Hide resolved
vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/lib/lmdblayer.py Outdated Show resolved Hide resolved
vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/lib/lmdbslab.py Outdated Show resolved Hide resolved
vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/lib/lmdbslab.py Outdated Show resolved Hide resolved
vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/lib/lmdbslab.py Outdated Show resolved Hide resolved
vEpiphyte
vEpiphyte reviewed Jun 3, 2019
View reviewed changes
synapse/lib/lmdbslab.py Outdated Show resolved Hide resolved
@vEpiphyte vEpiphyte modified the milestones: v0.1.10, v0.1.11 Jun 3, 2019
@jnwatson
Review fixes
d3d05cb 
'nomnommem' now 'dedicated' :(

Add statinfo method with locking statistics

lmdblayer now grabs the dedicated conf from cortex
invisig0th
invisig0th reviewed Jun 4, 2019
View reviewed changes
synapse/lib/lmdbslab.py Show resolved Hide resolved
synapse/lib/lmdbslab.py Show resolved Hide resolved
@jnwatson jnwatson merged commit e2c412c into master Jun 5, 2019
@jnwatson jnwatson deleted the vmtouch branch June 5, 2019 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vEpiphyte vEpiphyte vEpiphyte left review comments

@rakuy0 rakuy0 rakuy0 left review comments

@invisig0th invisig0th invisig0th approved these changes

Assignees

@jnwatson jnwatson

Labels
enhancement
Projects
None yet
Milestone
v0.1.11
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jnwatson @invisig0th @vEpiphyte @rakuy0