feat: shed: blockstore/vlog to car export cmds #8265
Conversation
Codecov Report
@@ Coverage Diff @@
## master #8265 +/- ##
==========================================
- Coverage 40.52% 40.29% -0.24%
==========================================
Files 675 676 +1
Lines 73524 73878 +354
==========================================
- Hits 29798 29771 -27
- Misses 38505 38880 +375
- Partials 5221 5227 +6
|
| Usage: "convert badger blockstore .vlog to .car", | ||
| Flags: []cli.Flag{ | ||
| &cli.PathFlag{ | ||
| Name: "vlog", |
There was a problem hiding this comment.
For my own curiosity: what is a badger vlog? I see a bunch of .vlog files under datastore repos. What is the meaning of each vlog subunit? Is it a partition of data based on the key for example?
There was a problem hiding this comment.
It's basically an append-only (afaiu) k-v write log. The Entry/header structs below tell you more exactly what is in those files
|
|
||
| var datastoreVlog2CarCmd = &cli.Command{ | ||
| Name: "vlog2car", | ||
| Usage: "convert badger blockstore .vlog to .car", |
There was a problem hiding this comment.
Any warnings needed about the daemon needing to be turned off as in the following utility?
There was a problem hiding this comment.
I'm not seeing any interaction with the repo lock file, any chance for problems if the node is online?
There was a problem hiding this comment.
vlog files are append-only, so that's (probably) fine
Just some tools to get data out of big badger datastores, and into car files.