io.file.writeAll does not append with StandardOpenOption.APPEND

[ryan-martin-axiell]

It seems that StandardOpenOption.APPEND is being ignored, instead io.file.writeAll will just overrwrite the file.

The below example results in two files; test-fs2.out and test-java.out. They both should contain the text:

a
b
c

However test-fs2.out only contains the letter c.

import cats.effect._

import fs2._
import fs2.io

import java.nio.file._

import scala.concurrent.ExecutionContext
import scala.language.higherKinds

object Scratch {
  def writeWithStream[T[_]](p: Path, s: String)
      (implicit F: ConcurrentEffect[T], cs: ContextShift[T]): T[Unit] = {
    Stream(s)
      .covary[T]
      .through(text.utf8Encode)
      .through(
        io.file.writeAll(
          p,
          scala.concurrent.ExecutionContext.global,
          Seq(StandardOpenOption.CREATE, StandardOpenOption.APPEND)
        )
      )
      .compile
      .drain
  }

  def main(args: Array[String]): Unit = {
    implicit val executionContext: ExecutionContext =
      scala.concurrent.ExecutionContext.Implicits.global

    implicit val contextShift: ContextShift[IO] =
      IO.contextShift(executionContext)

    val outPathfs2: Path = Paths.get("E:/Documents/test-fs2.out")
    val outPathJava: Path = Paths.get("E:/Documents/test-java.out")

    val fs2write = for {
      _ <- writeWithStream[IO](outPathfs2, "a\n")
      _ <- writeWithStream[IO](outPathfs2, "b\n")
      _ <- writeWithStream[IO](outPathfs2, "c\n")
    } yield ()

    fs2write.unsafeRunSync()

    Files.write(outPathJava, "a\n".getBytes(), StandardOpenOption.CREATE, StandardOpenOption.APPEND)
    Files.write(outPathJava, "b\n".getBytes(), StandardOpenOption.CREATE, StandardOpenOption.APPEND)
    Files.write(outPathJava, "c\n".getBytes(), StandardOpenOption.CREATE, StandardOpenOption.APPEND)

    println("done")
  }
}

[djspiewak]

This seems weird and concerning. Is it only on Windows? I haven't had a chance to reproduce it yet, but your test case is awesome.

[mpilquist]

I haven't looked to close but I think I've noticed this before -- writeAll calls _writeAllToFileHandle1 which takes an offset that's initialized to 0. Seems like if APPEND is specified, we should be initializing the offset to the size of the file instead of initializing to 0.

[mpilquist]

Fixed by #1381