Skip to content

Conversation

@Quafadas
Copy link
Contributor

@Quafadas Quafadas commented Sep 8, 2023

I think I've written this method a few times now, and keep having to scratch my head each time :-)...

So here it is - no pressure to merge if it isn't idiomatic :-).

Case classes to CSV
@zetashift
Copy link
Contributor

Actually I've needed this several times as well, thank you! :P. I like the pipe approach more personally.

I would like the examples to be a bit more "hey why you would do it and this is how" style, so I would phrase it more like this:

## Writing data to a CSV file

If you want to save a list of a case class into a CSV file, you could do the following:
@:select(scala-version)

@:choice(scala-3)

\```scala mdoc:reset:silent
import fs2.*
import fs2.data.csv.*
import fs2.io.file.Path

// create an encoder for your case class, by creating a `CsvRowEncoder[YourCaseClass, String]`
given CsvRowEncoder[YourCaseClass, String] = deriveCsvRowEncoder

def writeCaseClassToCsv[A](path: Path)(using CsvRowEncoder[A, String]): Pipe[IO, A, Nothing] =
    _.through(encodeUsingFirstHeaders(fullRows = true))
      .through(text.utf8Encode)
      .through(Files[IO].writeAll(path))
      .drain      
\```

@:choice(scala-2)

\```scala mdoc:reset:silent
import fs2._
import fs2.data.csv._
import fs2.io.file.Path

// create an encoder for your case class, by creating a `CsvRowEncoder[YourCaseClass, String]`
implicit val csvRowEncoder: CsvRowEncoder[YourCaseClass, String] = deriveCsvRowEncoder

def writeCaseClassToCsv[A](path: Path)(using CsvRowEncoder[A, String]): Pipe[IO, A, Nothing] =
    _.through(encodeUsingFirstHeaders(fullRows = true))
      .through(text.utf8Encode)
      .through(Files[IO].writeAll(path))
      .drain      

\```

@Quafadas
Copy link
Contributor Author

Quafadas commented Sep 9, 2023

With the benefit of hinsight, my original PR was rushed, and perhaps a little sloppy. Will tidy with the suggestions above - thanks for feedback - agreed on all points. Will revise in the next days when I have a few mins at keyboard to devote to it!

@Quafadas
Copy link
Contributor Author

Thanks for all the feedback - incorporated and hopefully mergable now :-)

Copy link
Contributor

@zetashift zetashift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to say that even if you might think this PR looked rushed, it still helps a lot with just getting the ball rolling. And that is always valuable (especially in open source).

edit: fml github ui bamboozled me, I thought I submitted my suggestions, anyhow see blow

Quafadas and others added 2 commits September 12, 2023 11:36
agreed

Co-authored-by: zetashift <rskaraya@gmail.com>
Co-authored-by: zetashift <rskaraya@gmail.com>
@zetashift
Copy link
Contributor

@Quafadas just the last 2 suggestions and besides that it LGTM!

@Quafadas
Copy link
Contributor Author

@zetashift I thought I already approved and committed them? Ah... I fear however, that I did not press the "resolve conversation" button ... gonna do that right now.

Quafadas and others added 2 commits September 21, 2023 09:47
Co-authored-by: zetashift <rskaraya@gmail.com>
Co-authored-by: zetashift <rskaraya@gmail.com>
@Quafadas
Copy link
Contributor Author

@zetashift Oh dear, I honestly thought I'd done them. I've had another go.

Copy link
Contributor

@zetashift zetashift left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thank you for this :D

Copy link
Member

@TonioGela TonioGela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a couple of comments in the code ;)
Thanks for your contribution!

@zetashift
Copy link
Contributor

zetashift commented Oct 7, 2023

Here's a runnable example:

//> using toolkit typelevel:latest
//> using scala 3.3.1
import fs2.io.file.Files

import fs2.data.csv.*
import fs2.data.csv.generic.semiauto.*
import fs2.io.file.Path
import cats.effect.{IO, IOApp}
import fs2.{Pipe, Stream}
import java.time.LocalDate

/** Define your case class and derive an encoder for it */
case class YourCaseClass(n: String, i: Int)
given CsvRowEncoder[YourCaseClass, String] = deriveCsvRowEncoder

/** This is our helper function that writes a case class as a csv given a path. */
def writeCaseClassToCsv[A](
    path: Path
)(using CsvRowEncoder[A, String]): Pipe[IO, A, Nothing] =
  _.through(encodeUsingFirstHeaders(fullRows = true))
    .through(fs2.text.utf8.encode)
    .through(Files[IO].writeAll(path))
    .drain

/** Let's imagine we have a `Book` case class we would like to write to
  * a .csv file.
  */
object WriteBooksToCsv extends IOApp.Simple:
  case class Book(id: Long, name: String, isbn: String)
  given CsvRowEncoder[Book, String] = deriveCsvRowEncoder

  val input = Seq(
    Book(1, "Programming in Scala", "9780997148008"),
    Book(2, "Hands-on Scala Programming", "9798387677205"),
    Book(3, "Functional Programming in Scala", "9781617299582")
  )

  def run: IO[Unit] =
    Stream
      .emits(input)
      .through(writeCaseClassToCsv(Path("books.csv")))
      .compile
      .drain *> IO.println("Finished writing books to books.csv.")

@Quafadas if you don't have time to update the PR, is it okay if I add this in? Would hate for this to sit too long.

Comment on lines +427 to +439
```scala
// Define your case class and derive an encoder for it
case class YourCaseClass(n: String, i: Int)
given CsvRowEncoder[YourCaseClass, String] = deriveCsvRowEncoder

// Writes a case class as a csv given a path.
def writeCaseClassToCsv[A](
path: Path
)(using CsvRowEncoder[A, String]): Pipe[IO, A, Nothing] =
_.through(encodeUsingFirstHeaders(fullRows = true))
.through(fs2.text.utf8.encode)
.through(Files[IO].writeAll(path))
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's scala 3 only, can you add the scala 2 tab too?

@zetashift
Copy link
Contributor

Continued here: #120 because completing this PR through Quafadas fork is getting too slow. Still @Quafadas thank you!!

@zetashift zetashift closed this Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants