[issue-67] unit tests for FlinkPravegaWriter #76
Conversation
- consolidate existing writer tests into FlinkPravegaWriterITCase - develop a test harness for sink functions - develop new unit tests for FlinkPravegaWriter - adjust checkstyle to allow the use of mockito - make internal fields and classes of FlinkPravegaWriter visible for testing - fix style check failures
| @@ -58,6 +58,10 @@ | |||
|
|
|||
| private static final long DEFAULT_TX_SCALE_GRACE_MILLIS = 10 * 60 * 1000; // 10 minutes | |||
|
|
|||
| // Writer interface to assist exactly-once and atleast-once functionality | |||
| private List<Integer> readAllEvents(final String streamName) throws Exception { | ||
| Preconditions.checkNotNull(streamName); | ||
|
|
||
| // TODO: Remove the end marker workaround once the following issue is fixed: |
There was a problem hiding this comment.
Pravega issue #408 has been resolved, but I wonder if we should use end of data. does it make sense to use the termination implemented here in pravega/pravega#1916 ?
| List<Integer> readElements = readAllEvents(streamName); | ||
|
|
||
| // Now verify that all expected events are present in the stream. Having extra elements are fine since we are | ||
| // testing the atleast once writer. |
| * Integration tests for {@link FlinkPravegaWriter}. | ||
| */ | ||
| @Slf4j | ||
| public class FlinkPravegaWriterITCase { |
There was a problem hiding this comment.
What does ITstands for in the name?
There was a problem hiding this comment.
Sorry, 'ITCase' is a convention in Flink to designate integration tests.
| verify(internalWriter).open(); | ||
|
|
||
| // verify that exceptions don't interfere with close | ||
| Mockito.doThrow(new IntentionalRuntimeException()).when(internalWriter).close(); |
There was a problem hiding this comment.
I'm not sure what this is doing. Is it the case that the first Mockito.doThrow will always throw, in which case the following call is not exercised? I only need a clarification of what the intention is here, please.
There was a problem hiding this comment.
The intention here is to exercise the try..catch blocks within FlinkPravegaWriter::close() that ensure that everything is definitely closed (by suppressing the exceptions). The mock statement causes the internal writer to throw upon close. Likewise the executor service is compelled to throw upon shutdown.
| Mockito.doThrow(new IntentionalRuntimeException()).when(context.executorService).shutdown(); | ||
| } | ||
| Assert.fail("expected an exception"); | ||
| } catch (IOException e) { |
There was a problem hiding this comment.
To clarify, what's suppose to be throwing IOException?
| .addSink(pravegaWriter).setParallelism(2); | ||
|
|
||
| final long executeStart = System.nanoTime(); | ||
| System.out.println(env.getExecutionPlan()); |
…into issue-67-2
Closes #67
Change log description
Purpose of the change
How to verify it
N/A