Need a mechanism to compress data to reduce storage costs

[laptrinhbockchain]

Currently the Base node need about additional 100G to store data every day. With data growth so fast, it causes huge obstacles in terms of infrastructure costs.

I think that need a mechanism to compress data to reduce storage costs. This helps many Nodes join the network to increase the decentralization of the project.

Currently, I don't know if this mechanism exists. But is there any way like Node configuration, or some solution to minimize this capacity. Such as:

1. Compress old data
2. Eliminate old data (Node only queries recent data)
3. Move old data to cheaper Storage like HDD
4. ...

[darren2025]

I also think it's quite necessary to act

[wbnns]

@laptrinhbockchain

Thanks for the feedback and for opening this issue. Yes, currently it's not possible to prune the node, which would decrease the required storage space.

Leaving this open and logging this feature request as I'm sure others would also like to have this option should it become available in the future.

[mdehoog]

You can run the op-geth with --gcmode=full (see #146) which will greatly reduce disk space requirements, but not allow you to read blocks older than 128 from chain tip.

We don't yet have a snapshot available for full nodes so it will take a while to sync. Will update this issue if and when we have full snapshots.

[HiroyukiNaito]

@mdehoog I see. Thank you!

[wangtao19911111]

@mdehoog ，are there any updates here?

[wbnns]

@wangtao19911111 Hello, full node snapshots are coming very soon.

[MindlessSteel]

What about cloud or Ai computation assistance
Computing transactions in the background is the fastest and efficient way to verify everything
Is legit, in my opinion

[kmalloc]

@wangtao19911111 Hello, full node snapshots are coming very soon.

any update? maintaining a full node these day no long an easy task.

[wbnns]

@laptrinhbockchain @MindlessSteel @wangtao19911111

Hi there! Just wanted to update that Base Node now supports snap sync by default! Simply:

1. docker compose stopyour Node
2. Pull the latest code from https://github.com/base-org/node
3. Make sure your ENVs have the correct L1 and Beacon endpoints, then
4. docker compose up -- build.

Whereas the current archive chain data is about 2.5TB, you'll be able to snap sync for less than 250GB.

If you all need any help or support, please let us know!

[adapt7]

@wbnns hi there. So I had fully synced full node but almost got out of disk space so I've just used your instruction above and set sync mode to snap and rebuilt the docker.

How long do I wait till geth data dir usage changes from 4TB to 0.25TB? It happens automatically I assume?

[roberto-bayardo]

Fastest way would be to just delete the old DB and start a fresh snap-sync. I am not sure it will go back in time and prune the old data automatically?

[fridary]

@laptrinhbockchain @MindlessSteel @wangtao19911111
Whereas the current archive chain data is about 2.5TB, you'll be able to snap sync for less than 250GB.

@wbnns why main github page says a high performance SSD drive with at least 4 TB free (NVME recommended) instead of 2.5TB?

[wbnns]

@fridary

Heya, this is factoring in room for the archive chain data to grow, plus unallocated space in the environment (in case it needs to be utilized).