Obsolete 0dns MongoDB

[NoSkillGuy]

[Cleanup] removing MongoDB since it's not required.

Why are we using MongoDB?

• To store the blocks

Are we using the stored Blocks in MongoDB?

• No
• Probably for some future use like
  • We can have an endpoint where we can give the complete chain
  • We can use the stored blocks to verify the chain

Philosophy :

• Sharders responsibility is to store blocks in a decentralized way.
• Miners responsibility is to verify the blocks before adding them to the chain in a decentralized way.

Pitfalls :

• If we are storing the complete chain, then we are depending on this single store. This defeats the purpose of the blockchain as this is a central system that can collapse

[Bug] when fetching magic block from shaders, we were fetching the block returned by the last sharder instead we should return the block which got max consensus.

• self explanatory, check here

[Code Quality] when we are starting a worker, we were starting a goroutine inside a goroutine, which is not required

• check here && here

[bbist]

Thanks for a well written issue.
Maybe we could impose such structure for all new issues.

Anyway, I think we need to consider a few things before finalizing linked PR.

Right now, 0dns is not implemented in a way to fully fulfill it's purpose. For example, it's supposed responsibility is to make available the latest and correct magic-block to all clients that desire it. What that means for a client is that it gets guarantee that it has the latest MB and it is the correct one. And once clients have a latest magic block, they can then start interacting with sharders for the MB chain verification.

Just as all stateful clients maintain valid magic block chain history, 0dns also needs to do it because we need to ensure 0dns is fault-tolerant. In practice, there will likely be multiple 0dns nodes to provide such tolerance. But when one specific node goes down and comes back after a while, it will interact with other 0dns nodes to get latest magic block, and verify magic-block chain history by tracing back the magic-block chain up until the latest one it has in it's own database before it went down. If we remove the database, the verification would require it to trace back to genesis magic block. So it may make sense to keep track of magic blocks that are already fetched and trusted. In case storing full blocks is redundant, we can instead choose to store only the magic-block headers (number, hash, prevhash, etc).

To summarize, we can either remove or keep the persistent db. Both choices have some advantage. If I had to choose I'd go for keeping trusted chain of historical MB headers, but it's not absolutely necessary and we can do the verification instead by fetching full chain from sharders.

@NoSkillGuy, @guruhubb please share your thoughts.

[NoSkillGuy]

Thanks for a well written issue.
Maybe we could impose such structure for all new issues.

✌🏻. I will create a v1 version of GitHub pull request template and let's observe and evolve with time
[update]: created a PR template - #10

Right now, 0dns is not implemented in a way to fully fulfill it's purpose. For example, it's supposed responsibility is to make available the latest and correct magic-block to all clients that desire it. What that means for a client is that it gets guarantee that it has the latest MB and it is the correct one. And once clients have a latest magic block, they can then start interacting with sharders for the MB chain verification.

Completely agree on this part

Just as all stateful clients maintain valid magic block chain history, 0dns also needs to do it because we need to ensure 0dns is fault-tolerant. In practice, there will likely be multiple 0dns nodes to provide such tolerance.

How I understood 0dns is, it should have the latest MB. The way we are ensuring that we have the latest MB is, we fetch it from sharders. Since we are using consensus to determine the MB, It should be valid and fault-tolerant. 0dns stores the MB in memory and tries to update the MB every 5 seconds.

[Scope for Improvement]: I think we should come up with an approach to actually get the latest MB in near real-time than depending on the 5-second retry

But when one specific node goes down and comes back after a while, it will interact with other 0dns nodes to get latest magic block, and verify magic-block chain history by tracing back the magic-block chain up until the latest one it has in it's own database before it went down. If we remove the database, the verification would require it to trace back to genesis magic block. So it may make sense to keep track of magic blocks that are already fetched and trusted. In case storing full blocks is redundant, we can instead choose to store only the magic-block headers (number, hash, prevhash, etc).

If one specific node goes down, when it comes back, all it needs is to fetch the latest MB from sharders right? That makes 0dns stateful.

To summarize, we can either remove or keep the persistent db. Both choices have some advantage. If I had to choose I'd go for keeping trusted chain of historical MB headers, but it's not absolutely necessary and we can do the verification instead by fetching full chain from sharders.

Having the history of all the block has its own advantages and disadvantages, But I think maintaining the history of blocks is not the responsibility of 0dns. We can release 0dns from maintaining history.

We can still create a side project, something like 0blocks, which will have history of all the MB's. But to make the system 0chain work in a decentralized way, 0blocks shouldn't be used for any core operations. Since this part works in a centralized fashion and can have all the disadvantages of scaling!

[NoSkillGuy]

@bbist let me know your thoughts on this!

[Sriep]

Looks good.