[BEAM-2311] Update compatibility matrix Aggregators section for Metrics #248
Conversation
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 9ea5300 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Looks good re: What Spark runner supports. The terms logical and physical may confuse users here, especially since the API refers to these by methods |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 6625d21 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
src/_data/capability-matrix.yml
Outdated
| values: | ||
| - class: model | ||
| l1: 'Partially' | ||
| l2: user-provided metrics | ||
| l3: Allow transforms to aggregate simple metrics across bundles in a <tt>DoFn</tt>. Semantically equivalent to using an additional output, but support partial results as the transform executes. Will likely want to augment <tt>Aggregators</tt> to be more useful for processing unbounded data by making them windowed. | ||
| l3: Allow transforms to gather simple metrics across bundles in a <tt>PTransform</tt>. Provide a mechanism to obtain both committed and attempted metrics. Semantically equivalent to using an additional output, but support partial results as the transform executes. Will likely want to augment <tt>Metrics</tt> to be more useful for processing unbounded data by making them windowed. |
There was a problem hiding this comment.
Committed metrics are semantically equivalent to using an additional output, right? Not attempted.
src/_data/capability-matrix.yml
Outdated
| l2: may miscount in streaming mode | ||
| l3: Current model is fully supported in batch mode. In streaming mode, <tt>Aggregators</tt> may under or overcount when bundles are retried. | ||
| l2: Gauge metric type is not supported. | ||
| l3: Current model is fully supported for Counter and Distribution metrics. |
There was a problem hiding this comment.
Metrics do not work in streaming mode?
|
Addressed Dan's comments. |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id e00ffd5 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
src/_data/capability-matrix.yml
Outdated
| - class: dataflow | ||
| l1: 'Partially' | ||
| l2: Gauge metric type is not supported. | ||
| l3: Current model is fully supported for Counter and Distribution metrics. | ||
| l2: Batch supports committed Counters and Distributions. No streamming support. |
| - class: dataflow | ||
| l1: 'Partially' | ||
| l2: may miscount in streaming mode | ||
| l3: Current model is fully supported in batch mode. In streaming mode, <tt>Aggregators</tt> may under or overcount when bundles are retried. | ||
| l2: Batch supports committed Counters and Distributions. No streamming support. |
There was a problem hiding this comment.
add link to JIRA, with some text like "coming soon [BEAM-XXX]"
r: @dhalperi @aviemzur @JingsongLi
Feel free to suggest rephrasing of the descriptions. I'll incorporate it.