Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[stateless_validation] Integration with in-memory trie #10564

Closed
Tracked by #46
Longarithm opened this issue Feb 2, 2024 · 2 comments
Closed
Tracked by #46

[stateless_validation] Integration with in-memory trie #10564

Longarithm opened this issue Feb 2, 2024 · 2 comments
Assignees
@staffik
Labels
A-stateless-validation Area: stateless validation

Comments

@Longarithm
Copy link
Member

Enable in-memory trie for chunk producers. For that, implementation of shard switch is required.
@Longarithm Longarithm added the A-stateless-validation Area: stateless validation label Feb 2, 2024
@Longarithm Longarithm mentioned this issue Feb 2, 2024
Open
52 tasks
@staffik staffik mentioned this issue Feb 8, 2024
Closed
7 tasks
@github-actions github-actions bot mentioned this issue Mar 1, 2024
Closed
@walnut-the-cat
Copy link
Contributor

@staffik , let's update this task as we make progress

This was referenced Mar 14, 2024
Closed
Merged
github-merge-queue bot pushed a commit that referenced this issue Mar 27, 2024
@staffik
Enable in-memory trie when state sync (#10820)
d4d1b82 
**Issue**: #10564

**Summary**
Adds logic to load / unload in-memory tries that works with state sync.
Enables in-memory trie with single shard tracking.

**Changes**
- Add optional `state_root` parameter for memtrie loading logic - it's
needed when we cannot read the state root from chunk extra.
- Add `load_mem_tries_for_tracked_shards` config parameter.
- Add methods for loading / unloading in-memory tries.
- Remove obsolete tries from memory before each new state sync.

**Follow up tasks**
- Add shard assignment shuffling every epoch for StatelessNet:
#10845.
- Make sure the logic works well with resharding, and state is fully
GC-ed, add integration tests:
#10844.
@walnut-the-cat
Copy link
Contributor

can we close the issue if completed? cc. @staffik

@staffik staffik closed this as completed Apr 5, 2024
@staffik staffik mentioned this issue Apr 8, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@staffik staffik

Labels
A-stateless-validation Area: stateless validation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Longarithm @staffik @walnut-the-cat