[SPARK-3887] Send stracktrace in ConnectionManager error replies

[JoshRosen]

When reporting that a remote error occurred, the ConnectionManager should also log the stacktrace of the remote exception. This PR accomplishes this by sending the remote exception's stacktrace as the payload in the "negative ACK / error message."

[JoshRosen]

Thanks to @patmcdonough for suggesting this. /cc @rxin for review.

[JoshRosen]

I suppose that this could break if we added another message type and used it to report errors. Do you think that I should pre-emptively add error handling code around this?

[SparkQA]

QA tests have started for PR 2741 at commit cef18b3.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2741 at commit cef18b3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21561/Test PASSed.

[zsxwing]

It's really helpful 👍 Just a small question: Should specify charset when converting between bytes and String, or Spark always assumes the environment uses UTF8?

[rxin]

Changes lgtm overall.

[JoshRosen]

Should specify charset when converting between bytes and String, or Spark always assumes the environment uses UTF8?

It looks like String.getBytes uses the platform's default charset, so it's probably safer to explicitly specify a charset. I've updated the patch to do this.

[SparkQA]

QA tests have started for PR 2741 at commit b5366cc.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2741 at commit b5366cc.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21598/Test PASSed.

[zsxwing]

LGTM. Is it worth to add val UTF8 = Charset.forName("UTF-8") into object Utils?

[rxin]

It is probably good to add, but maybe in a separate PR?

[rxin]

I'm merging this in master.