Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Don't keep all blocks in memory #675

Closed
theoreticalbts opened this issue Dec 7, 2016 · 4 comments
Closed

Don't keep all blocks in memory #675

theoreticalbts opened this issue Dec 7, 2016 · 4 comments
Assignees
@mvandeberg

Comments

@theoreticalbts
Copy link
Contributor

Currently a fair amount of the memory usage of steemd is due to keeping all the blocks in memory. It's fairly low-hanging fruit to take this data out of memory and read from the block log instead.
@mvandeberg mvandeberg self-assigned this Dec 7, 2016
@mvandeberg
Copy link
Contributor

mvandeberg commented Dec 7, 2016 •
edited
Loading

I have been working on this a little, but if you have some extra time to get it done, go for it.

@BhuzOr
Copy link

BhuzOr commented Dec 7, 2016

#650

mvandeberg pushed a commit that referenced this issue Dec 7, 2016
When blocks are written to log, they are removed from chain state mem…
51f99ca 
…ory #675
@mvandeberg mvandeberg mentioned this issue Dec 7, 2016
Closed
mvandeberg pushed a commit that referenced this issue Dec 8, 2016
Fix broken tests #675
e21d608
@mvandeberg mvandeberg mentioned this issue Dec 8, 2016
Merged
@arhag
Copy link
Contributor

arhag commented Dec 8, 2016

Why are you storing blocks into ChainBase at all in the first place? Even if you remove that data later, that is still causing additional pages of memory to become dirty unnecessarily, which may lead to additional IO. There is no need for the packed_block field in the block_stats_object. Blocks newer than the last irreversible block can be retrieved from _fork_db in the fetch_block_by_number function.

mvandeberg pushed a commit that referenced this issue Dec 12, 2016
Removed packed_block #675
f409f75
mvandeberg pushed a commit that referenced this issue Dec 13, 2016
Remove packed_block #675
adae614
mvandeberg pushed a commit that referenced this issue Dec 13, 2016
Remove block_stats_object #675
d4ba29d
@mvandeberg mvandeberg mentioned this issue Dec 13, 2016
Closed
@mvandeberg
Copy link
Contributor

Merged

mvandeberg pushed a commit that referenced this issue Dec 15, 2016
Fix block log error #675
3effcc2
mvandeberg pushed a commit that referenced this issue Dec 16, 2016
Block num fork resolution when writing to block log #675
4c05914
mvandeberg pushed a commit that referenced this issue Dec 16, 2016
Fix block log error #675
993c06f
mvandeberg pushed a commit that referenced this issue Dec 16, 2016
Block num fork resolution when writing to block log #675
551062f
@theoreticalbts theoreticalbts mentioned this issue Dec 19, 2016
Open
mvandeberg pushed a commit that referenced this issue Jan 2, 2017
Add extra assertions to detect a corrupt block log #738 #675
77ee892
@mvandeberg mvandeberg mentioned this issue Jan 3, 2017
Merged
On1x pushed a commit to VIZ-Blockchain/viz-cpp-node that referenced this issue May 28, 2018
@afalaleev
Merge pull request steemit#675 from GolosChain/674-optional-reputation
e36e8a3 
Optional field reputation. steemit#674
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mvandeberg mvandeberg

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@BhuzOr @arhag @theoreticalbts @mvandeberg