SPARK-5613: Catch the ApplicationNotFoundException exception to avoid thread from getting killed on yarn restart.

[kasjain]

[SPARK-5613] Added a catch block to catch the ApplicationNotFoundException. Without this catch block the thread gets killed on occurrence of this exception. This Exception occurs when yarn restarts and tries to find an application id for a spark job which got interrupted due to yarn getting stopped.
See the stacktrace in the bug for more details.

[AmplabJenkins]

Can one of the admins verify this patch?

[andrewor14]

ok to test

[andrewor14]

This change looks reasonable. LGTM once tests pass

[SparkQA]

Test build #26926 has started for PR 4392 at commit 4831000.

• This patch merges cleanly.

[andrewor14]

Hey @kasjain can you open this against the master branch next time? It will be easier for us to back port stuff from there

[SparkQA]

Test build #26926 has finished for PR 4392 at commit 4831000.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26926/
Test PASSed.

[andrewor14]

Merging into 1.2 thanks.

[andrewor14]

Hey @kasjain this is already merged into branch 1.2. Would you mind closing this?

[kasjain]

Sure. Thanks :)

From: andrewor14 <notifications@github.commailto:notifications@github.com>
Reply-To: apache/spark <reply@reply.github.commailto:reply@reply.github.com>
Date: Saturday, February 7, 2015 at 1:22 AM
To: apache/spark <spark@noreply.github.commailto:spark@noreply.github.com>
Cc: Kashish Jain <kashish.jain@guavus.commailto:kashish.jain@guavus.com>
Subject: Re: [spark] SPARK-5613: Catch the ApplicationNotFoundException exception to avoid thread from getting killed on yarn restart. (#4392)

Hey @kasjainhttps://github.com/kasjain can you open this against the master branch next time? It will be easier for us to back port stuff from there

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4392#issuecomment-73301321.

[kasjain]

Thanks for closing this issue.

From: andrewor14 <notifications@github.commailto:notifications@github.com>
Reply-To: apache/spark <reply@reply.github.commailto:reply@reply.github.com>
Date: Saturday, February 7, 2015 at 3:19 AM
To: apache/spark <spark@noreply.github.commailto:spark@noreply.github.com>
Cc: Kashish Jain <kashish.jain@guavus.commailto:kashish.jain@guavus.com>
Subject: Re: [spark] SPARK-5613: Catch the ApplicationNotFoundException exception to avoid thread from getting killed on yarn restart. (#4392)

Hey @kasjainhttps://github.com/kasjain this is already merged into branch 1.2. Would you mind closing this?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4392#issuecomment-73318823.

[pwendell]

@andrewor14 this was never merged into branch 1.3 and I believe that caused a recent build break. So I just merged it.

[lianhuiwang]

i think for yarn-cluster mode, we also should catch the ApplicationNotFoundException for getApplicationReport.@pwendell @andrewor14

[kasjain]

Hi,

Is this change required in yarn-cluster mode as well? Is yes, is there any already opened ticket for this?

Thanks
Kashish

From: Lianhui Wang <notifications@github.commailto:notifications@github.com>
Reply-To: apache/spark <reply@reply.github.commailto:reply@reply.github.com>
Date: Thursday, February 12, 2015 at 1:15 PM
To: apache/spark <spark@noreply.github.commailto:spark@noreply.github.com>
Cc: Kashish Jain <kashish.jain@guavus.commailto:kashish.jain@guavus.com>
Subject: Re: [spark] SPARK-5613: Catch the ApplicationNotFoundException exception to avoid thread from getting killed on yarn restart. (#4392)

i think for yarn-cluster mode, we also should catch the ApplicationNotFoundException.@pwendellhttps://github.com/pwendell@andrewor14https://github.com/andrewor14

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4392#issuecomment-74030105.