Skip to content

Commit

Permalink
[SPARK-24507][DOCUMENTATION] Update streaming guide
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?
Updated streaming guide for direct stream and link to integration guide.

## How was this patch tested?
jekyll build

Author: Rekha Joshi <rekhajoshm@gmail.com>

Closes #21683 from rekhajoshm/SPARK-24507.

(cherry picked from commit f599cde)
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
  • Loading branch information
rekhajoshm authored and HyukjinKwon committed Jul 2, 2018
1 parent 3c0af79 commit 1cba050
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/streaming-programming-guide.md
Expand Up @@ -2176,6 +2176,8 @@ the input data stream (using `inputStream.repartition(<number of partitions>)`).
This distributes the received batches of data across the specified number of machines in the cluster
before further processing.

For direct stream, please refer to [Spark Streaming + Kafka Integration Guide](streaming-kafka-integration.html)

### Level of Parallelism in Data Processing
{:.no_toc}
Cluster resources can be under-utilized if the number of parallel tasks used in any stage of the
Expand Down

0 comments on commit 1cba050

Please sign in to comment.