Skip to content

v0.2.7

Choose a tag to compare

View all tags
@nevillelyh nevillelyh released this 21 Nov 17:44
v0.2.7
33b815f

"Crescat scientia vita excolatur"

Notable changes:

  • Allow user to set job name with dataflow runner #341
  • Delay creating local output directories until after sc.close() #321
  • Support custom IO in JobTest #317

ScioResult#saveMetrics now saves dataflow service metrics in addition to accumulators and other basic job metrics.

BigQueryType.fromTable now keeps the pre-formatted table string in the generated companion object so that the table name can be formatted at runtime. This is useful for, for example, BigQuery tables partitioned by date, where a fixed value for the date is used in the annotation to supply the schema, but the actual table queried at runtime is different. This is now at parity with BigQueryType.fromQuery.

ScioContext#applyTransform and SCollection#applyOutputTransform have been replaced with ScioContext#customInput and SCollection#saveAsCustomOutput.

Assets 3
Loading