[SPARK-12058][STREAMING][KINESIS][TESTS] fix Kinesis python tests#10050
[SPARK-12058][STREAMING][KINESIS][TESTS] fix Kinesis python tests#10050brkyvz wants to merge 5 commits intoapache:masterfrom
Conversation
|
Test build #46910 has finished for PR 10050 at commit
|
|
@brkyvz could you enable |
|
jenkins retest this please |
|
Test build #46926 has finished for PR 10050 at commit
|
|
jenkins, test this please |
|
jenkins test this please |
|
Test build #47091 has finished for PR 10050 at commit
|
|
Test build #47090 has finished for PR 10050 at commit
|
There was a problem hiding this comment.
Lets name it KinesisDataGenerator, to avoid confusion with "producer" in KPL.
KinesisProducer --> KinesisDataGenerator
KinesisClientProducer --> SimpleDataGenerator
KinesisProducerLibraryProducer --> KPLDataGenerator
|
Test build #47102 has finished for PR 10050 at commit
|
|
Test build #2169 has finished for PR 10050 at commit
|
|
Test build #2170 has finished for PR 10050 at commit
|
|
LGTM. Merging this to master and 1.6. Thanks! |
Python tests require access to the `KinesisTestUtils` file. When this file exists under src/test, python can't access it, since it is not available in the assembly jar. However, if we move KinesisTestUtils to src/main, we need to add the KinesisProducerLibrary as a dependency. In order to avoid this, I moved KinesisTestUtils to src/main, and extended it with ExtendedKinesisTestUtils which is under src/test that adds support for the KPL. cc zsxwing tdas Author: Burak Yavuz <brkyvz@gmail.com> Closes #10050 from brkyvz/kinesis-py.
Python tests require access to the
KinesisTestUtilsfile. When this file exists under src/test, python can't access it, since it is not available in the assembly jar.However, if we move KinesisTestUtils to src/main, we need to add the KinesisProducerLibrary as a dependency. In order to avoid this, I moved KinesisTestUtils to src/main, and extended it with ExtendedKinesisTestUtils which is under src/test that adds support for the KPL.
cc @zsxwing @tdas