Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-9012][WebUI]Escape Accumulators in the task table #7369

Closed
wants to merge 1 commit into from
Closed

[SPARK-9012][WebUI]Escape Accumulators in the task table #7369

wants to merge 1 commit into from

Conversation

zsxwing
Copy link
Member

@zsxwing zsxwing commented Jul 13, 2015

If running the following codes, the task table will be broken because accumulators aren't escaped.

val a = sc.accumulator(1, "<table>")
sc.parallelize(1 to 10).foreach(i => a += i)

Before this fix,

screen shot 2015-07-13 at 8 02 44 pm

After this fix,

screen shot 2015-07-13 at 8 14 32 pm

@SparkQA
Copy link

SparkQA commented Jul 13, 2015

Test build #37134 has started for PR 7369 at commit a83c9b6.

@shaneknapp
Copy link
Contributor

i'll retrigger this build once the jenkins system updates are completed

@squito
Copy link
Contributor

squito commented Jul 13, 2015

lgtm

@shaneknapp
Copy link
Contributor

jenkins test this please

@SparkQA
Copy link

SparkQA commented Jul 13, 2015

Test build #37138 has finished for PR 7369 at commit a83c9b6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class Least(children: Expression*) extends Expression
    • case class Greatest(children: Expression*) extends Expression

@andrewor14
Copy link
Contributor

Jenkins, slow test please

@JoshRosen
Copy link
Contributor

LGTM.

@SparkQA
Copy link

SparkQA commented Jul 14, 2015

Test build #2 has finished for PR 7369 at commit a83c9b6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member

sarutak commented Jul 15, 2015

LGTM as well. Merging into master and branch-1.4.

asfgit pushed a commit that referenced this pull request Jul 15, 2015
If running the following codes, the task table will be broken because accumulators aren't escaped.
```
val a = sc.accumulator(1, "<table>")
sc.parallelize(1 to 10).foreach(i => a += i)
```

Before this fix,

<img width="1348" alt="screen shot 2015-07-13 at 8 02 44 pm" src="https://cloud.githubusercontent.com/assets/1000778/8649295/b17c491e-299b-11e5-97ee-4e6a64074c4f.png">

After this fix,

<img width="1355" alt="screen shot 2015-07-13 at 8 14 32 pm" src="https://cloud.githubusercontent.com/assets/1000778/8649337/f9e9c9ec-299b-11e5-927e-35c0a2f897f5.png">

Author: zsxwing <zsxwing@gmail.com>

Closes #7369 from zsxwing/SPARK-9012 and squashes the following commits:

a83c9b6 [zsxwing] Escape Accumulators in the task table

(cherry picked from commit adb33d3)
Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
@asfgit asfgit closed this in adb33d3 Jul 15, 2015
@zsxwing zsxwing deleted the SPARK-9012 branch July 15, 2015 08:45
mingyukim pushed a commit to palantir/spark that referenced this pull request Sep 22, 2015
If running the following codes, the task table will be broken because accumulators aren't escaped.
```
val a = sc.accumulator(1, "<table>")
sc.parallelize(1 to 10).foreach(i => a += i)
```

Before this fix,

<img width="1348" alt="screen shot 2015-07-13 at 8 02 44 pm" src="https://cloud.githubusercontent.com/assets/1000778/8649295/b17c491e-299b-11e5-97ee-4e6a64074c4f.png">

After this fix,

<img width="1355" alt="screen shot 2015-07-13 at 8 14 32 pm" src="https://cloud.githubusercontent.com/assets/1000778/8649337/f9e9c9ec-299b-11e5-927e-35c0a2f897f5.png">

Author: zsxwing <zsxwing@gmail.com>

Closes apache#7369 from zsxwing/SPARK-9012 and squashes the following commits:

a83c9b6 [zsxwing] Escape Accumulators in the task table

(cherry picked from commit adb33d3)
Signed-off-by: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants