Skip to content

Commit

Permalink
Adding docs for spark.serializer.objectStreamReset configuration
Browse files Browse the repository at this point in the history
  • Loading branch information
kellrott committed Feb 27, 2014
1 parent 7ccc74b commit f70d069
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions docs/configuration.md
Expand Up @@ -237,6 +237,17 @@ Apart from these, the following properties are also available, and may be useful
exceeded" exception inside Kryo. Note that there will be one buffer <i>per core</i> on each worker.
</td>
</tr>
<tr>
<td>spark.serializer.objectStreamReset</td>
<td>10000</td>
<td>
When serializing using org.apache.spark.serializer.JavaSerializer, the serializer caches
objects to prevent writing redundant data, however that stops garbage collection of those
objects. By calling 'reset' you flush that info from the serializer, and allow old
objects to be collected. To turn off this periodic reset set it to a value of <= 0.
By default it will reset the serializer every 10,000 objects.
</td>
</tr>
<tr>
<td>spark.broadcast.factory</td>
<td>org.apache.spark.broadcast.<br />HttpBroadcastFactory</td>
Expand Down

0 comments on commit f70d069

Please sign in to comment.