Skip to content

Commit

Permalink
修正《3.2》中缺失的图片
Browse files Browse the repository at this point in the history
  • Loading branch information
lw-lin committed Nov 5, 2016
1 parent d36b9c2 commit 4c4d2fb
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -131,16 +131,14 @@ def onStart() {

我们看到在上面的代码中,executor 端会先 `new` 一个 `ReceiverSupervisorImpl`,然后 `ReceiverSupervisorImpl.start()`。这里 `.start()` 很重要的工作就是调用 `Receiver.onStart()`,来启动 `Receiver` 的数据接收线程:

![image](3.imgs/050.JPG)
![image](3.imgs/050.png)

`start()` 成功后,`ReceiverSurpervisorImpl` 最重要的工作就是接收 `Receiver``store()` 过来的数据了。

`ReceiverSurpervisorImpl` 有 4 种签名的 `push()` 方法,被 `Receiver` 的 4 种 `store()` 一一调用。不过接下来对单条小数据和三种块数据的处理稍有区别。

单条的情况,`ReceiverSupervisorImpl` 要在 `BlockGenerator` 的协助下,将多个单条的数据积攒为一个块数据,然后重新调用 `push` 交给 `ReceiverSurpervisorImpl` 来处理这个块数据。我们一会再详解 `BlockGenerator` 的这个过程。

![image](3.imgs/055.JPG)

所以接下来,我们主要看这 3 个存储块数据的 `push...()` 方法,它们的实现非常简单:

```scala
Expand Down Expand Up @@ -170,7 +168,7 @@ def pushAndReportBlock(receivedBlock: ReceivedBlock, ...) {

上面的过程可以总结为:

![image](3.imgs/055.JPG)
![image](0.imgs/065.png)

## ReceivedBlockHandler 详解

Expand Down Expand Up @@ -269,7 +267,7 @@ def storeBlock(blockId: StreamBlockId, block: ReceivedBlock): ReceivedBlockStore

`BlockGenerator` 工作的整个过程示意图如下:

![image](3.imgs/060.png)
![image](3.imgs/060.png) *//TODO(lwlin): 此图风格与本系列文章不符,需要美化*

## 总结

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,8 @@

`ReceivedBlockTracker` 内部,有几个重要的成员,它们的关系如下:

![image](3.imgs/070.png)
![image](3.imgs/070.png) *//TODO(lwlin): 此图风格与本系列文章不符,需要美化*


- `streamIdToUnallocatedBlockQueues`
- 维护了上报上来的、但尚未分配入 batch 的 `Block` 块数据的 meta
Expand Down Expand Up @@ -201,7 +202,7 @@

这 4 个方法,和对应信息状态的修改关系如下图总结:

![image](3.imgs/075.png)
![image](3.imgs/075.png) *//TODO(lwlin): 此图风格与本系列文章不符,需要美化*

上面即是 `ReceivedBlockTracker` 的主体内容。

Expand Down
Binary file added Spark Streaming 源码解析系列/3.imgs/050.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 4c4d2fb

Please sign in to comment.