Skip to content

Version 1.3.0
Compare
Choose a tag to compare
@falaki falaki released this 20 Nov 20:14
· 33 commits to master since this release
v1.3.0
84858ce

Spark-csv 1.3.0 adds following

Features:

  • Printing content of records when failing or dropping
  • Support for Timestamps in type inference
  • Support for roundtrip null values of any type (by @andy327)
  • Parsing double and float according to Locale if default cast fails (by @gasparms)
  • Support for nullable quote character (by @jamesblau)
  • Using system line separator for parsing (by Glenn Murray)
  • Support for pruned scan for required fields (by @HyukjinKwon)
  • Support for compressionCodec in all languages (by @msperlich)
  • Support for data source short name with Spark 1.5+

Improvements

  • Several documentation updates and fixes (by multiple community contributors)
  • Several code style and compatibility automated tests (by @JoshRosen)
  • Unit test consolidation (by @brkyvz)
Assets 2