Conversation
The post goes into details on the previous and expanded testing infrastructure provided by the Beam SDK and provides examples of the times at which elements arrive, and how that impacts the outputs produced by a PipelineRunner.
tgroh
force-pushed
the
test_stream_blog_post
branch
from
ec8f638 to
ef0c5ce
Compare
|
I haven't yet gotten everything to render locally, fwiw |
francesperry
left a comment
francesperry
left a comment
There was a problem hiding this comment.
Few quick thoughts just on the markdown. Let me know when you've had a chance to check the rendering yourself and I'll do a more detailed review.
_posts/2016-10-17-test-stream.md
Outdated
| title: "Testing Unbounded Pipeliens in Apache Beam" | ||
| date: 2016-10-17 10:21:48 -0800 | ||
| excerpt_separator: <!--more--> | ||
| categories: blog beam testing java |
There was a problem hiding this comment.
Don't add all these. They make the URL weird. I'd just stick to 'blog'. Also filed BEAM-776 to figure out how to make this not ugly.
There was a problem hiding this comment.
Done.
(Picture me collapsing at the knees and cradling my head)
| excerpt_separator: <!--more--> | ||
| categories: blog beam testing java | ||
| authors: | ||
| - tgroh |
There was a problem hiding this comment.
Need to add yourself to _data/authors.yml
_posts/2016-10-17-test-stream.md
Outdated
| primitives provide a way for users to perform useful, powerful, and correct | ||
| computations in spite of these challenges. | ||
|
|
||
| We need comprehensive tests that cover crucial failure scenarios and corner |
There was a problem hiding this comment.
Perhaps preface with "As users of Beam, we..." to clarify this isn't referring to testing Beam itself.
_posts/2016-10-17-test-stream.md
Outdated
| unit tests for pipelines that may receive late data or trigger multiple times | ||
| has historically ranged from complex to not possible, as pipelines that read | ||
| from unbounded sources do not shut down without external intervention, while | ||
| Pipelines that read from bounded sources exclusively cannot test behavior with |
_posts/2016-10-17-test-stream.md
Outdated
| that use custom triggers and handle out-of-order data could not be easily tested. | ||
|
|
||
| This blog post introduces our new framework for writing tests for pipelines that | ||
| handle delayed and out-of-order data in the context of the LeaderBoard/Mobile |
There was a problem hiding this comment.
Leaderboard pipeline from the Mobile Gaming example series.
tgroh
force-pushed
the
test_stream_blog_post
branch
2 times, most recently
from
00d1a25 to
ebc54c0
Compare
|
Generated static content. Fixed spelling of title. I will need to update the post date before merge, including content regeneration. |
|
Please don't regenerate content/ -- just leave that whole directory out and let the person doing the final merge do it. Also, see new shiny website instructions. Also, you'll want to rename the file to the date you expect to publish. 10/20 seem reasonable? |
tgroh
force-pushed
the
test_stream_blog_post
branch
from
ebc54c0 to
883d4bc
Compare
|
Gone has the content. Moved everything over. |
tgroh
force-pushed
the
test_stream_blog_post
branch
from
883d4bc to
3170dd2
Compare
tgroh
force-pushed
the
test_stream_blog_post
branch
3 times, most recently
from
c5e369b to
e4cdf42
Compare
|
Looks good to me. Merging. |
2 participants
The post goes into details on the previous and expanded testing
infrastructure provided by the Beam SDK and provides examples of the
times at which elements arrive, and how that impacts the outputs
produced by a PipelineRunner.