Filter deposit contract logs before pruning #9123
Conversation
eth/stagedsync/stage_log_index.go
Outdated
| @@ -175,6 +177,9 @@ func promoteLogIndex(logPrefix string, tx kv.RwTx, start uint64, endBlock uint64 | |||
| } | |||
|
|
|||
| for _, l := range ll { | |||
| if cfg.noPruneContracts != nil && !cfg.noPruneContracts[l.Address] && l.BlockNumber < pruneBlock { | |||
There was a problem hiding this comment.
for performance maybe better place l.BlockNumber < pruneBlock before map check
erigon-lib/chain/chain_config.go
Outdated
| @@ -82,12 +82,15 @@ type Config struct { | |||
| Clique *CliqueConfig `json:"clique,omitempty"` | |||
| Aura *AuRaConfig `json:"aura,omitempty"` | |||
| Bor *BorConfig `json:"bor,omitempty"` | |||
|
|
|||
| //For not pruning these contracts (e.g. depsoit contract logs needed by CL) | |||
There was a problem hiding this comment.
let's add here more comments. if you know more information now. like: "why CL need it" and "in which mode (archive/non-archive) CL need it" and maybe something else.
There was a problem hiding this comment.
Actually, I am only aware that both modes need it initially for constructing deposit data. Deeper dive into CL logic may be needed for me. And here I am just trying to preserve the logs, by default, irrespective of the specification of prune= flag.
I must run a validator test for this, although I am getting the desired results through RPC, the CL may have more expectations that I am not aware of.
There are also talks about Deposit snapshots which, then, would make this whole jutsu pointless
|
this looks correct. the only thing I am wary of is that this means there will be a silent failure if there is an issue with the logic that deletes more than it should. cl does a simple eth_getLogs, so if it's missing information it will just return as if there is nothing and continue without knowing better I don't think it's a problem, but something to think about. maybe there is a simple solution for this |
So there is no integrity check on the constructed state on CL side for deposits? |
not that I can see. what will happen is someone using those receipts to validate would miss the deposits and not include them in their blocks I believe. |
|
@somnathb1 |
Both |
somnathb1
changed the title
This is a hack to not prune logs of certain special contracts, designated through chain config property, `noPruneContracts` **Background** CL needs deposit contract logs to create and maintain its own state (deposit tree, in particular). It requests the data through `eth_getLogs`. The naive approach is to not prune any logs of any contract since the block of the deposit contract. This results in a much bloated database for Logs. A validating node must be like that. **Proposed solution** Add the deposit contract to a special list. On execute stage, skip pruning Log table. In LogIndex stage, selectively prune logs, i.e. don't prune those of any of the special contracts' logs. **Results** Gnosis pruned node went from about 600GB to 320GB, and can still be a validating node (not battle tested yet)
This is a hack to not prune logs of certain special contracts, designated through chain config property,
noPruneContractsBackground
CL needs deposit contract logs to create and maintain its own state (deposit tree, in particular). It requests the data through
eth_getLogs. The naive approach is to not prune any logs of any contract since the block of the deposit contract. This results in a much bloated database for Logs. A validating node must be like that.Proposed solution
Add the deposit contract to a special list. On execute stage, skip pruning Log table. In LogIndex stage, selectively prune logs, i.e. don't prune those of any of the special contracts' logs.
Results
Gnosis pruned node went from about 600GB to 320GB, and can still be a validating node (not battle tested yet)