Skip to content

Commit

Permalink
[SPARK-6391][DOCS] Document Tachyon compatibility.
Browse files Browse the repository at this point in the history
Adds a section in the RDD persistence section of the programming-guide docs detailing Spark-Tachyon version compatibility as discussed in [[SPARK-6391]](https://issues.apache.org/jira/browse/SPARK-6391).

Author: Calvin Jia <jia.calvin@gmail.com>

Closes #6382 from calvinjia/spark-6391 and squashes the following commits:

113e863 [Calvin Jia] Move compatibility info to the offheap storage level section.
7942dc5 [Calvin Jia] Add a section in the programming-guide docs for Tachyon compatibility.
  • Loading branch information
calvinjia authored and rxin committed May 25, 2015
1 parent 8af1bf1 commit ce0051d
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions docs/programming-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -1214,9 +1214,11 @@ storage levels is:
Compared to MEMORY_ONLY_SER, OFF_HEAP reduces garbage collection overhead and allows executors
to be smaller and to share a pool of memory, making it attractive in environments with
large heaps or multiple concurrent applications. Furthermore, as the RDDs reside in Tachyon,
the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory
the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory
in Tachyon is discardable. Thus, Tachyon does not attempt to reconstruct a block that it evicts
from memory.
from memory. If you plan to use Tachyon as the off heap store, Spark is compatible with Tachyon
out-of-the-box. Please refer to this <a href="http://tachyon-project.org/master/Running-Spark-on-Tachyon.html">page</a>
for the suggested version pairings.
</td>
</tr>
</table>
Expand Down

0 comments on commit ce0051d

Please sign in to comment.