Merge #4358

4358: Fix possible incomplete snapshot chunk write r=Zelldon a=Zelldon

## Description

As written in #4357 we need to loop on writing otherwise we may end in cases where your snapshot is corrupted because chunks have been written not completely.

<!-- Please explain the changes you made here. -->
Related guide to write to FileChannel http://tutorials.jenkov.com/java-nio/file-channel.html

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #4357 

#

Co-authored-by: Christopher Zell <zelldon91@googlemail.com>

broker/src/main/java/io/zeebe/broker/clustering/atomix/storage/snapshot/DbPendingSnapshot.java

@@ -100,7 +100,22 @@ public void write(final ByteBuffer chunkId, final ByteBuffer chunkData) {
 
     try (final var channel =
         Files.newByteChannel(path, StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE)) {
-      channel.write(chunkData);
+      final var expectedToWrite = chunkData.remaining();
+      long actualWrittenBytes = 0L;
+      while (chunkData.hasRemaining()) {
+        actualWrittenBytes += channel.write(chunkData);
+      }
+
+      if (actualWrittenBytes != expectedToWrite) {
+        throw new IllegalStateException(
+            "Expected to write "
+                + expectedToWrite
+                + " bytes of the given snapshot chunk with id "
+                + chunkId
+                + ", but only "
+                + actualWrittenBytes
+                + " bytes were written");
+      }
     } catch (final FileAlreadyExistsException e) {
       LOGGER.debug("Chunk {} of pending snapshot {} already exists at {}", filename, this, path, e);
     } catch (final IOException e) {