Skip to content

1. How to start?

@dawrutowicz edited this page Jun 8, 2019 · 1 revision

cleanframes provides basic transformations for primitives.

All these Cleaner type-class instances are defined in cleanframes.instances package.

Import these instances:

import cleanframes.instances.all._

Define model:

case class Model(col1: Option[Int],
                 col2: Option[Byte],
                 col3: Option[Short],
                 col4: Option[Long],
                 col5: Option[Float],
                 col6: Option[Double],
                 col7: Option[Boolean])

Library uses field names of case class to resolve DataFrame column names. Thus, there is a need to redefine column names:

val frame = input
  .toDF("col1", "col2", "col3", "col4", "col5", "col6", "col7")

To call API, add following import:

import cleanframes.syntax._ 

It enables to call API on a DataFrame instance:

frame.clean[Model]

Full running code example can be found here.