Skip to content

Commit

Permalink
Verified documentation update by building via jekyll
Browse files Browse the repository at this point in the history
  • Loading branch information
Ilya Ganelin committed Jan 9, 2015
1 parent 33b5a2d commit 3a38db1
Showing 1 changed file with 10 additions and 10 deletions.
20 changes: 10 additions & 10 deletions docs/programming-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -1316,31 +1316,31 @@ For accumulator updates performed inside <b>actions only</b>, Spark guarantees t
will only be applied once, i.e. restarted tasks will not update the value. In transformations, users should be aware
of that each task's update may be applied more than once if tasks or job stages are re-executed.

In addition, accumulators do not maintain lineage for the operations that use them. Consequently, accumulators updates are not guaranteed to be executed when made within a lazy transformation like `map()`. Unless something has triggered the evaluation of the lazy transformation that updates the value of the accumlator, subsequent operations will not themselves trigger that evaluation and the value of the accumulator will remain unchanged. The below code fragment demonstrates this issue:
In addition, accumulators do not maintain lineage for the operations that use them. Consequently, accumulator updates are not guaranteed to be executed when made within a lazy transformation like `map()`. Unless something has triggered the evaluation of the lazy transformation that updates the value of the accumlator, subsequent operations will not themselves trigger that evaluation and the value of the accumulator will remain unchanged. The below code fragment demonstrates this issue:

<div class="codetabs">

<div data-lang="scala" markdown="1">
{% highlight scala %}
val acc = sc.accumulator(0)
data.map(x => acc += x; f(x))
// Here, acc is still 0 because no actions have cause the `map` to be computed.
val acc = sc.accumulator(0)
data.map(x => acc += x; f(x))
// Here, acc is still 0 because no actions have cause the `map` to be computed.
{% endhighlight %}
</div>

<div data-lang="java" markdown="1">
{% highlight java %}
Accumulator<Integer> accum = sc.accumulator(0);
data.map(x -> accum.add(x); f(x););
// Here, acc is still 0 because no actions have cause the `map` to be computed.
Accumulator<Integer> accum = sc.accumulator(0);
data.map(x -> accum.add(x); f(x););
// Here, accum is still 0 because no actions have cause the `map` to be computed.
{% endhighlight %}
</div>

<div data-lang="python" markdown="1">
{% highlight python %}
accum = sc.accumulator(0)
data.map(lambda x => acc.add(x); f(x))
# Here, acc is still 0 because no actions have cause the `map` to be computed.
accum = sc.accumulator(0)
data.map(lambda x => acc.add(x); f(x))
# Here, acc is still 0 because no actions have cause the `map` to be computed.
{% endhighlight %}
</div>

Expand Down

0 comments on commit 3a38db1

Please sign in to comment.