Skip to content
Itto-CSV is a pure scala library for working with the CSV format
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
project 0.0.3 Jan 8, 2019
src
.bumpversion.cfg
.codecov.yml
.gitignore
.scalafmt.conf
.travis.yml
LICENSE
README.md
build.sbt
publish.sbt

README.md

Itto-CSV Build Status codecov.io Javadocs Maven Central Cats friendly

Itto-CSV is a pure scala library for working with the CSV format

Add the library to your project

libraryDependencies += "com.github.gekomad" %% "itto-csv" % "0.1.0"

Using Library

Formatters

The formatter determines how CSV will be generated

case class IttoCSVFormat(
    delimeter: Char,
    quote: Char,
    recordSeparator: String,
    quoteEmpty: Boolean,
    forceQuote: Boolean,
    printHeader: Boolean,
    trim: Boolean,
    ignoreEmptyLines: Boolean,
    quoteLowerChar: Boolean)

Two formatters are available:

Method Description Default Formatter Tab formatter
withDelimiter(Char) the separator between fields , \t
withQuote(Char) the quoteChar character " "
withQuoteEmpty(Boolean) quotes field if empty false false
withForceQuote(Boolean) quotes all fields false false
withPrintHeader(Boolean) if true prints the header (method toCsvL) false false
withTrim(Boolean) trims the field false false
withRecordSeparator(String) the rows separator \r\n \r\n
withIgnoreEmptyLines(Boolean) skips empty lines false false
withQuoteLowerChar(Boolean) quotes lower chars false false

It's possible to create custom formatters editing the default ones, example: implicit val newFormatter = default.withForceQuote(true).withRecordSeparator("\n").with.....

Use the library

Types

You can use defined types or you can define yours

Type to CSV

Trasforming a case class to CSV

import com.github.gekomad.ittocsv.core.ToCsv._
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default

case class Bar(a: String, b: Int)
assert(toCsv(Bar("侍", 42)) == "侍,42")
case class Baz(x: String)
case class Foo(a: Int, c: Baz)
case class Xyz(a: String, b: Int, c: Foo)

assert(toCsv(Xyz("hello", 3, Foo(1, Baz("hi, dude")))) == "hello,3,1,\"hi, dude\"")
assert(toCsv(List(1.1, 2.1, 3.1)) == "1.1,2.1,3.1")

CSV to List of Type

Trasforming a CSV to a list of case class

import com.github.gekomad.ittocsv.core.FromCsv._
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default

case class Bar(a: String, b: Int)
assert(fromCsv[Bar]("abc,42") == List(Right(Bar("abc", 42))))
assert(fromCsv[Bar]("abc,42\r\nfoo,24") == List(Right(Bar("abc", 42)), Right(Bar("foo", 24))))

case class Foo(v: String, a: List[Int])
assert(fromCsv[Foo]("abc,\"1,2,3\"") == List(Right(Foo("abc", List(1, 2, 3)))))

CSV to List

trasform a CSV to a list of case class

import com.github.gekomad.ittocsv.core.FromCsv._
import com.github.gekomad.ittocsv.core.ParseFailure
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default

assert(fromCsvL[Double]("1.1,2.1,3.1") == List(Right(1.1), Right(2.1), Right(3.1)))
assert(fromCsvL[Double]("1.1,abc,3.1") == List(Right(1.1), Left(ParseFailure("abc is not Double")), Right(3.1)))

List of Type to CSV

Trasforming a list of case class to a CSV multirows

import com.github.gekomad.ittocsv.core.ToCsv._
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default

case class Bar(a: String, b: Int)
assert(toCsvL(List(Bar("Bar", 42),Bar("Foo", 24))) == "a,b\r\nBar,42\r\nFoo,24")

Read from file


import com.github.gekomad.ittocsv.parser.io.FromFile.csvFromFile
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.tab

case class Bar(id: String, name: String, date: String)

val path = "/tmp/csv_with_header.csv"
val list = csvFromFile[Bar](path, skipHeader = true) // Try[Seq[Either[NonEmptyList[ParseFailure], Bar]]]

Write to file

case class Bar(id: String, name: String)

import com.github.gekomad.ittocsv.parser.io.ToFile.csvToFile
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.tab.withPrintHeader(true).withRecordSeparator("\n")
import com.github.gekomad.ittocsv.core.ToCsv._

val list = List(Bar("A1","Jack"),Bar("A2","Bob"))

val filePath = "/tmp/out.csv"
csvToFile(list, filePath)

Get Header

It's possible to get the header starting from the chosen class

case class Foo(i: Int, d: Double, s: Option[String], b: Boolean)
import com.github.gekomad.ittocsv.core.Header._

{
  implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default
  assert(csvHeader[Foo] == "i,d,s,b")
}

{
  implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default.withDelimiter('|').withForceQuote(true)
  assert(csvHeader[Foo] == """"i"|"d"|"s"|"b"""")
}

Encode/Decode some types

You can change the regex pattern with

implicit val emailValidator: Validator[Email] = com.github.gekomad.ittocsv.core.Types.implicits.validatorEmail.copy(regex = yourRegex)

import java.util.UUID
import com.github.gekomad.ittocsv.core.Types.implicits._
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default

case class Bar(a: String, b: SHA1, c: SHA256, d: MD5, e: UUID, f: Email, g: IP6, h: BitcoinAdd, i: URL)

val sha1       = SHA1("1c18da5dbf74e3fc1820469cf1f54355b7eec92d")
val uuid       = UUID.fromString("1CC3CCBB-C749-3078-E050-1AACBE064651")
val md5        = MD5("23f8e84c1f4e7c8814634267bd456194")
val sha256     = SHA256("000020f89134d831f48541b2d8ec39397bc99fccf4cc86a3861257dbe6d819d1")
val email      = Email("daigoro@itto.com")
val ip         = IP6("2001:db8:a0b:12f0::1")
val bitcoinAdd = BitcoinAdd("3Nxwenay9Z8Lc9JBiywExpnEFiLp6Afp8v")
val url        = URL("http://www.aaa.cdd.com")

val bar = Bar("abc", sha1, sha256, md5, uuid, email, ip, bitcoinAdd, url)

val csvString =
  "abc,1c18da5dbf74e3fc1820469cf1f54355b7eec92d,000020f89134d831f48541b2d8ec39397bc99fccf4cc86a3861257dbe6d819d1,23f8e84c1f4e7c8814634267bd456194,1cc3ccbb-c749-3078-e050-1aacbe064651,daigoro@itto.com,2001:db8:a0b:12f0::1,3Nxwenay9Z8Lc9JBiywExpnEFiLp6Afp8v,http://www.aaa.cdd.com"

//encode
import com.github.gekomad.ittocsv.core.ToCsv._
assert(toCsv(bar) == csvString)

//decode
import com.github.gekomad.ittocsv.core.FromCsv._
assert(fromCsv[Bar](csvString) == List(Right(bar)))

Encode/Decode your own type

Using encode for MyType:

implicit def _e(implicit csvFormat: IttoCSVFormat): CsvStringEncoder[MyType] = ???

Example encoding N:Int to "[N]"

import com.github.gekomad.ittocsv.core.CsvStringEncoder
import com.github.gekomad.ittocsv.parser.IttoCSVFormat
implicit val csvFormat: IttoCSVFormat = IttoCSVFormat.default

case class MyType(a: Int)
case class Foo(a: MyType, b: Int)

import com.github.gekomad.ittocsv.core.ToCsv._

implicit def _e(implicit csvFormat: IttoCSVFormat): CsvStringEncoder[MyType] = createEncoder { node =>
  csvConverter.stringToCsvField(s"[${node.a}]")
}

assert(toCsv(Foo(MyType(42),99)) == "[42],99")

Using decode for MyType:

implicit def _d(implicit csvFormat: IttoCSVFormat): String => Either[ParseFailure, MyType] = (str: String) => ???

Example decoding "[N]" to N:Int

import com.github.gekomad.ittocsv.parser.IttoCSVFormat
import scala.util.Try
implicit val csvFormat: IttoCSVFormat = IttoCSVFormat.default
import com.github.gekomad.ittocsv.core.{ParseFailure}
import cats.data.NonEmptyList

case class MyType(a: Int)
case class Foo(a: MyType, b: Int)

import com.github.gekomad.ittocsv.core.FromCsv._

implicit def _d(implicit csvFormat: IttoCSVFormat): String => Either[ParseFailure, MyType] = { (str: String) =>
  if (str.startsWith("[") && str.endsWith("]"))
    Try(str.substring(1, str.size - 1).toInt)
      .map(f => Right(MyType(f)))
      .getOrElse(Left(ParseFailure(s"Not a MyType $str")))
  else Left(ParseFailure(s"Wrong format $str"))

}

assert(fromCsv[Foo]("[42],99") == List(Right(Foo(MyType(42),99))))
assert(fromCsv[Foo]("[x],99") == List(Left(NonEmptyList(ParseFailure("Not a MyType [x]"), Nil))))
assert(fromCsv[Foo]("42,99") == List(Left(NonEmptyList(ParseFailure("Wrong format 42"), Nil))))

The TreeTest.scala shows how to encode/decode a Tree[Int]

CSV to List of type with LocalDateTime

Trasforming a CSV string to a class list; if the trasformations fails resturns a Left with the cause of the error

import com.github.gekomad.ittocsv.parser.IttoCSVFormat
import com.github.gekomad.ittocsv.core.FromCsv._
import com.github.gekomad.ittocsv.core.Conversions.fromStringToLocalDateTime
case class Foo(a: Int, b: java.time.LocalDateTime)

implicit val csvFormat: IttoCSVFormat = IttoCSVFormat.default

val o = fromCsv[Foo]("1,2000-12-31T11:21:19")
assert(o == List(Right(Foo(1, java.time.LocalDateTime.parse("2000-12-31T11:21:19", java.time.format.DateTimeFormatter.ISO_LOCAL_DATE_TIME)))))

CSV to List of type with custom LocalDateTime

Trasforming a CSV string to a class list; if the trasformations fails resturns a Left with the cause of the error

import com.github.gekomad.ittocsv.parser.IttoCSVFormat
import com.github.gekomad.ittocsv.core.ParseFailure
import com.github.gekomad.ittocsv.core.FromCsv._

case class Foo(a: Int, b: java.time.LocalDateTime)

import java.time.LocalDateTime
import java.time.format.DateTimeFormatter

implicit val csvFormat = IttoCSVFormat.default

implicit def localDateTimeToCsv: String => Either[ParseFailure, LocalDateTime] = {
 case s => scala.util.Try {
   Right(LocalDateTime.parse(s, DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.0")))
 }.getOrElse(Left(ParseFailure(s"Not a LocalDataTime $s")))
}

val o = fromCsv[Foo]("1,2000-12-31 11:21:19.0")
assert(o == List(Right(Foo(1, LocalDateTime.parse("2000-12-31 11:21:19.0", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.0"))))))

List of type with LocalDateTime to CSV


import com.github.gekomad.ittocsv.core.ToCsv._
import java.time.LocalDateTime
import com.github.gekomad.ittocsv.core.CsvStringEncoder
import java.time.format.DateTimeFormatter
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default.withPrintHeader(false)

val myFormatter=DateTimeFormatter.ofPattern("yyyy ** MM ** dd HH ++ mm ++ ss")

implicit def localDateTimeEncoder(implicit csvFormat: com.github.gekomad.ittocsv.parser.IttoCSVFormat): CsvStringEncoder[LocalDateTime] =
 (value: LocalDateTime) => value.format(myFormatter)

case class Bar(a: String, b: Long, c: java.time.LocalDateTime, e: Option[Int])

val myDate = java.time.LocalDateTime.parse("2000 ** 12 ** 31 11 ++ 21 ++ 19", myFormatter)

val l: List[Bar] = List(Bar("Yel,low", 3L, myDate, Some(1)), Bar("eee", 7L, myDate, None))

assert(toCsvL(l) == "\"Yel,low\",3,2000 ** 12 ** 31 11 ++ 21 ++ 19,1\r\neee,7,2000 ** 12 ** 31 11 ++ 21 ++ 19,")

List of type with custom LocalDateTime to CSV

import java.time.LocalDateTime
import com.github.gekomad.ittocsv.core.CsvStringEncoder
import java.time.format.DateTimeFormatter
import com.github.gekomad.ittocsv.core.ToCsv._
implicit val csvFormat = com.github.gekomad.ittocsv.parser.IttoCSVFormat.default.withPrintHeader(false)

implicit def localDateTimeEncoder(implicit csvFormat: com.github.gekomad.ittocsv.parser.IttoCSVFormat): CsvStringEncoder[LocalDateTime] =
  (value: LocalDateTime) => value.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.0"))

val localDateTime = LocalDateTime.parse("2000-11-11 11:11:11.0", DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.0"))

case class Bar(a: String, b: Long, c: LocalDateTime, e: Option[Int])
val l: List[Bar] = List(
  Bar("Yel,low", 3L, localDateTime, Some(1)),
  Bar("eee", 7L, localDateTime, None)
)
assert(toCsv(l) == "\"Yel,low\",3,2000-11-11 11:11:11.0,1,eee,7,2000-11-11 11:11:11.0,")

Spooling CSV file using FS2 Stream and Doobie

See Doobie Recepies project

Further examples

From CSV to Type

From Type To CSV

Defined types

Email

Ciphers

  • UUID (1CC3CCBB-C749-3078-E050-1AACBE064651)
  • MD5 (23f8e84c1f4e7c8814634267bd456194)
  • SHA1 (1c18da5dbf74e3fc1820469cf1f54355b7eec92d)
  • SHA256 (000020f89134d831f48541b2d8ec39397bc99fccf4cc86a3861257dbe6d819d1)

URL, IP, MAC Address

  • IP (10.192.168.1)
  • IP_6 (2001:db8:a0b:12f0::1)
  • URLs (http://abc.def.com)
  • Youtube (https://www.youtube.com/watch?v=9bZkp7q19f0)
  • Facebook (https://www.facebook.com/thesimpsons - https://www.facebook.com/pages/)
  • Twitter (https://twitter.com/rtpharry)
  • MAC Address (fE:dC:bA:98:76:54)

HEX

  • HEX (#F0F0F0 - 0xF0F0F0)

Bitcoin

Phone numbers

Date time

Crontab

Codes

Concurrency

Strings

Logs

  • Apache error ([Fri Dec 16 02:25:55 2005] [error] [client 1.2.3.4] Client sent malformed Host header)

Numbers

Coordinates

Scaladoc API

Scala doc

Bugs and Feedback

For bugs, questions and discussions please use Github Issues.

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

You can’t perform that action at this time.