Skip to content

Import snapshot directly into hotstore for nodes running splitstore #10554

@ZenGround0

Description

@ZenGround0
Contributor

Checklist

  • This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
    I have a specific, actionable, and well motivated feature request to propose.

Lotus component

  • lotus daemon - chain sync
    lotus fvm/fevm - Lotus FVM and FEVM interactions
    lotus miner/worker - sealing
    lotus miner - proving(WindowPoSt/WinningPoSt)
    lotus JSON-RPC API
    lotus message management (mpool)
    Other

What is the motivation behind this feature request? Is your feature request related to a problem? Please describe.

Coldstore shouldn't track unwanted state in discard mode. Today we waste ~100 G of disk space no matter what when syncing from snapshot

Describe the solution you'd like

import-snaphsot daemon flag should import the chain into the hotstore if the node is configured to run with splitstore.

Basically repo should create a blockstore using hotstore instead of universal store location if applicable. We might want to make a new blockstore location that automatically does this or we could manually check the repo config during import.

Describe alternatives you've considered

We could also make it easier to purge the coldstore of snapshot state by improving chain prune cold-compact but it would be better to prevent this problem in the first place than to keep it around and also create a solution

Additional context

No response

Activity

added this to the Splitstore Enhancements milestone on May 12, 2023
removed this from the Splitstore Enhancements milestone on May 19, 2023
ZenGround0

ZenGround0 commented on Dec 25, 2024

@ZenGround0
ContributorAuthor

Looking at this more today the problem seems to be that

  1. Node startup depends on initial state in universal blockstore
  2. Splitstore startup depends on node startup

So we have to unravel these dependencies if we want to save the disk space

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Participants

      @TippyFlitsUK@ZenGround0@jennijuju

      Issue actions

        Import snapshot directly into hotstore for nodes running splitstore · Issue #10554 · filecoin-project/lotus