Batching of metrics

[akakitani]

Implement batching of metrics, so that when we want to collect metrics at the record level, the driver doesn't slow down with too many metrics being sent to it.

My initial idea for this is to have some configurable batching interval of x seconds. This interval can be configured by the user to match how often some external monitoring service checks these Spark metrics, so that even time-sensitive ones can remain relatively accurate.

With Spark 2.0, we should also explore using the newer accumulator APIs to see if we can accomplish this.

[BDeus]

Do you have an implementation or a begin of implementation of batching ?
Use spark-metrics at record level has a too high footprint on performance.