Prevent deletes on the bigchain table

[r-marques]

Deletes are not permitted in the bigchain table. We need a changefeed to listen to delete in the bigchain table and revert them

[ttmc]

Alarms should also go off, somehow, because something that should not have happened happened.

Which node would revert the delete? I suppose any node could do that. It would look like an insert, so we need to make sure the other nodes don't reject that insert. (Maybe there is nothing to do: the insertion of a block in the bigchain table is one of the allowed changes.)

It should be okay if multiple nodes revert the delete. One revert (insert) will happen first, and all the other ones will look like an attempt to write a document with the same primary key (block id): RethinkDB should reject those inserts.

Maybe all nodes could attempt to revert the delete. That's easy to implement. Deletion-reversions should be rare, so we don't need to think about the cost.

[sbellem]

How much of a challenge is disabling DELETE at the rethinkdb level?

Assuming doing the work on the RethinkDB C++ core is too much work, what about making a fork of the Python driver, and disabling delete at that level? That does not seem too much of a challenge at first sight.

See https://github.com/rethinkdb/rethinkdb/blob/bb735c63c7560bc6737656fd504068ff56917c5a/drivers/python/rethinkdb/ast.py#L307 for instance.

[r-marques]

@sbellem you have no way to enforce the usage of the forked / modified python driver. Nothing prevents a node of using the default python driver.

A federation node will have full access to the database so there is no way to prevent them from deleting data.

The best solution for now would be to use consensus for any database change like we already use on other parts of the system.

A fork of rethinkdb itself would be something to consider in the future after we know exactly what can and cannot be done with a "stock" rethinkdb.

[ttmc]

In any case, BigchainDB still needs the DELETE operation to delete transactions from the backlog table.

[jvsteiner]

Even if a customized rethinkdb fork is created and used, there are no guarantees that a given node, participating in consensus is running the customized version. I think it's always possible for a rogue node to maintain it's own version of truth as long as it can pretend to act correctly during the consensus process.

[ttmc]

BigchainDB doesn't use a customized RethinkDB fork; it uses plain vanilla RethinkDB and there are no plans to change that any time soon.

The node behavior you describe is known as a "Byzantine fault." RethinkDB's consensus protocol (Raft) can tolerate crash faults and similar (so long as less than half the nodes are affected), but not Byzantine faults.