@adelbertc adelbertc released this Oct 15, 2017 · 184 commits to master since this release

Assets 2

Notable additions/changes:

  • support for Spark 2.2.0
  • added Encoders for: UDT, Array, Map
  • added explode() on TypedColumns with types Vector/Array
  • added bitwise and/or/xor operators on TypedColumns
  • added withColumn() operator on TypedDataset
  • added pivot() aggregation
  • added statistical methods: corr(), skewness(), kurtosis(), cover_sample()
  • migrated from SparkContext to SparkSessions throughout
  • created the frameless-ml project
  • parameterize Spark actions over the effect type used (a much more powerful Job[_])
  • [Internal] Improved test template
  • [Internal] Moved methods that are not optimizable by Catalyst to a new package (map(), flatMap(), etc.)
  • [bug] Fixed bug in computing equality of nullable types (Options)
  • [bug] Fixed big-decimal devision incorrectly returning Double