Skip to content

Commit

Permalink
modified streaming test case to add coment
Browse files Browse the repository at this point in the history
  • Loading branch information
giwa committed Aug 18, 2014
1 parent 99e4bb3 commit 580fbc2
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions python/pyspark/streaming_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@
"""
Unit tests for PySpark; additional tests are implemented as doctests in
individual modules.
Other option is separate this test case with other tests.
This makes sense becuase streaming tests takes long time due to waiting time
for stoping callback server.
This file will merged to tests.py. But for now, this file is separated due
to focusing to streaming test case
Expand Down Expand Up @@ -45,7 +48,7 @@ def tearDown(self):
self.ssc._sc.stop()
# Why does it long time to terminaete StremaingContext and SparkContext?
# Should we change the sleep time if this depends on machine spec?
time.sleep(8)
time.sleep(10)

@classmethod
def tearDownClass(cls):
Expand Down Expand Up @@ -302,7 +305,7 @@ def _run_stream(self, test_input, test_func, expected_output, numSlices=None):
"""Start stream and return the output"""
# Generate input stream with user-defined input
numSlices = numSlices or self.numInputPartitions
test_input_stream = self.ssc._testInputStream2(test_input, numSlices)
test_input_stream = self.ssc._testInputStream(test_input, numSlices)
# Apply test function to stream
test_stream = test_func(test_input_stream)
# Add job to get output from stream
Expand Down

0 comments on commit 580fbc2

Please sign in to comment.