Skip to content

v0.7.0-alpha1

Pre-release
Pre-release

Choose a tag to compare

@nevillelyh nevillelyh released this 20 Sep 06:11

Breaking changes

  • See v0.7.0 Migration Guide for detailed instructions
  • New Magnolia based Coders derivation replaces ClassTag and Kryo
  • New ScioIO replaces TestIO[T] to simplify IO implementation and stubbing in JobTest
  • Update dynamic file destination API #1305
  • Remove deprecated TensorFlow graph prediction method #1370
  • Object file IO is no longer backwards compatible due to coder changes

Features

  • Magnolia & macro based coder for magnitudes faster (de)serialization
  • Redesigned unified ScioIO[T] for all IO modules
  • Add SCollection#{readAll,readAllBytes} (splittable DoFn support) #796 #1363
  • Add sparse left and right outer joins #1386
  • Check and warn chained joins #1362
  • Support Parquet compression #1189 #1318
  • Port Parquet IO to Parquet 1.10 #1340 #1345
  • Configurable fetch and batch size for JDBC IO #1314

Bug fixes