Usefulness of batch

[reynir]

I was recently reviewing code by @dinosaure that implements Mirage_kv.RW, and there were some differences between the implementation and the behavior prescribed by the documentation in mirage-kv (see the thread).

The documentation is very precise in describing the behavior of batch which I think in general is a good thing. However, I think the behavior can be difficult to achieve in many implementations. It also mandates a default retries value of 13 - which may be undesirably high in many implementations. There seems to be a contradiction as well: It says:

Concurrent operations will not affect other ones executed during the batch.

And then the next line says

Batch applications can fail to apply if other operations are happening concurrently.

I am not sure how to read this, but if concurrent operations do not affect others during a batch then they shouldn't make the batch fail, or?

mirage-kv/src/mirage_kv.mli

Lines 251 to 264 in c66cf9d

	val batch: t -> ?retries:int -> (t -> 'a Lwt.t) -> 'a Lwt.t
	(** [batch t f] run [f] in batch. Ensure the durability of
	operations.
	Since a batch is applied at once, the readings inside a batch
	will return the state before the entire batch. Concurrent
	operations will not affect other ones executed during the batch.
	Batch applications can fail to apply if other operations are
	happening concurrently. In case of failure, [f] will run again
	with the most recent version of [t]. The result is
	[Error `Too_many_retries] if [f] is run for more then [retries] attemps
	(default is [13]). *)

I reviewed a couple of implementations found through opam list --depends-on mirage-kv: fat-filesystem, chamelon, tar-mirage, irmin-mirage-git and git-kv (which was pinned on my computer)

• Ocaml-fat only implements KV_RO interface despite having write support. I don't know why the fat implementation doesn't implement RW, but I imagine it would be difficult to implement batch as described. Otherwise I think implementing RW should be easy. https://github.com/mirage/ocaml-fat/blob/6d675256565f9818ca21043e7d7f388a3f4fae36/src/fat.mli#L93-L110

• Chamelon implements a 'noop' batch (i.e. calling batch kv f and f kv is equivalent): https://github.com/yomimono/chamelon/blob/13b08cb7926855a7be8bc9a3787e56395a925506/mirage/kv.ml#L189-L193

• Tar-mirage implements as well a 'noop' batch: https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mirage.ml#L434

• Irmin-mirage-git may be closest to implementing the described semantics (I am not sure, I don't fully understand the behavior from a quick read of the code), but notably it defaults to 42 retries instead of 13. I suspect irmin-mirage-git motivated adding batch to mirage-kv. https://github.com/mirage/irmin/blob/3dd2231f7d1d7d409ae4704486183d49f79e7095/src/irmin-mirage/git/irmin_mirage_git.ml#L297-L339

Since no one seems to implement batch exactly as described I think we should consider changing or loosening the requirements of implementations or just remove batch altogether.

[hannesm]

Thanks for raising your concerns. in the previous mirage meeting we settled to remove batch from the interface (done in #37). Any kv implementation can just extend the interface with that functionality.