r/consensus: do not allow follower to truncate log that is consumable

If an offset was already visible a follower must not be allowed to
truncate it as it may lead to a situation in which an offset is visible
and not readable.

Visible batches has the same replication guarantees as committed batches
as leader still waits for the majority to acknowledge message at given
offset before making it visible to the readers. This makes it possible
not to truncate offsets which were previously visible.

Signed-off-by: Michal Maslanka <michal@redpanda.com>

src/v/raft/consensus.cc

@@ -2008,11 +2008,11 @@ consensus::do_append_entries(append_entries_request&& r) {
 
     // section 3
     if (request_metadata.prev_log_index < last_log_offset) {
-        if (unlikely(request_metadata.prev_log_index < _commit_index)) {
+        if (unlikely(request_metadata.prev_log_index < last_visible_index())) {
             reply.result = reply_result::success;
             // clamp dirty offset to the current commit index not to allow
             // leader reasoning about follower log beyond that point
-            reply.last_dirty_log_index = _commit_index;
+            reply.last_dirty_log_index = last_visible_index();
             reply.last_flushed_log_index = _commit_index;
             vlog(
               _ctxlog.info,