Includes Splittable DoFn in the capability matrix #259
Conversation
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id adf7f2b with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
retest this please |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id adf7f2b with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
The test failure looks pre-existing. |
|
I agree it is not related. Curiously, that link is not dead via the live site. It is the Maturity Model's "550 JIRA resolved during incubation" link, if I haven't made a mistake. |
|
It seems like it might be nice to give a ~2 sentence summary of the support when details are expanded. For example, the sorts of splitting that we do. It is redundant with other documentation we'll need to provide, but adds to the ubiquity of documentation that leads to users just knowing what to do I think. |
|
Added some more information, also to the Source API. |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id d9eeb79 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
Ping. |
|
retest this please |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id d9eeb79 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
src/_data/capability-matrix.yml
Outdated
| values: | ||
| - class: model | ||
| l1: 'Partially' | ||
| l2: DoFn with non-monolithic element processing |
There was a problem hiding this comment.
Can we reverse the polarity here and describe non-monolothic instead? Something like "splitting and checkpointing of processing within an element"? My phrasing is not great, but something...
src/_data/capability-matrix.yml
Outdated
| - class: model | ||
| l1: 'Partially' | ||
| l2: DoFn with non-monolithic element processing | ||
| l3: Allows users to develop DoFn's that process a single element in portions ("restrictions"), executed in parallel or sequentially, making it possible to use DoFn for developing bounded and unbounded data sources much more flexible and composable with the rest of the Beam model than the Source API. See http://s.apache.org/splittable-do-fn. Design is in progress on achieving parity with Source API regarding progress signals. |
There was a problem hiding this comment.
I think keeping it more descriptive and pithy rather than just relating to the Source API. Maybe
Allows users to develop DoFn's that process a single element in portions ("restrictions"), executed in parallel or sequentially. This supersedes the unbounded and bounded `Source` APIs by supporting all of their features on a per-element basis. See http://s.apache.org/splittable-do-fn. Design is in progress on achieving parity with Source API regarding progress signals.
|
Thanks, PTAL. |
|
@asfbot merge |
|
Refer to this link for build results (access rights to CI server needed): Jenkins built the site at commit id 3cc42f8 with Jekyll and staged it here. Happy reviewing. Note that any previous site has been deleted. This staged site will be automatically deleted after its TTL expires. Push any commit to the pull request branch or re-trigger the build to get it staged again. |
|
@asfgit merge |
R: @kennknowles