Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-6490][Docs] Add docs for rpc configurations #5607

Closed
wants to merge 5 commits into from
Closed

[SPARK-6490][Docs] Add docs for rpc configurations #5607

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1c2cf26
Add docs for rpc configurations
zsxwing Apr 21, 2015
4f07174
Fix unit tests
zsxwing Apr 21, 2015
5577540
Use spark.network.timeout as the default timeout if it presents
zsxwing Apr 21, 2015
6e37c30
Update docs
zsxwing Apr 21, 2015
25a6736
Increase the default timeout to 120s
zsxwing Apr 22, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion core/src/test/scala/org/apache/spark/SparkConfSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ class SparkConfSuite extends FunSuite with LocalSparkContext with ResetSystemPro
test("akka deprecated configs") {
val conf = new SparkConf()

assert(!conf.contains("spark.rpc.num.retries"))
assert(!conf.contains("spark.rpc.numRetries"))
assert(!conf.contains("spark.rpc.retry.wait"))
assert(!conf.contains("spark.rpc.askTimeout"))
assert(!conf.contains("spark.rpc.lookupTimeout"))
Expand Down
2 changes: 1 addition & 1 deletion core/src/test/scala/org/apache/spark/rpc/RpcEnvSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ abstract class RpcEnvSuite extends FunSuite with BeforeAndAfterAll {

val conf = new SparkConf()
conf.set("spark.rpc.retry.wait", "0")
conf.set("spark.rpc.num.retries", "1")
conf.set("spark.rpc.numRetries", "1")
val anotherEnv = createRpcEnv(conf, "remote", 13345)
// Use anotherEnv to find out the RpcEndpointRef
val rpcEndpointRef = anotherEnv.setupEndpointRef("local", env.address, "ask-timeout")
Expand Down
28 changes: 28 additions & 0 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -982,6 +982,34 @@ Apart from these, the following properties are also available, and may be useful
This is only relevant for the Spark shell.
</td>
</tr>
<tr>
<td><code>spark.rpc.numRetries</code></td>
<td>3</td>
How many times for an RPC ask operation to retry before giving up.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Number of times to retry before an RPC task gives up.

(We should also indicate whether 1 = retry once, or 1 = run it once in total)

<td>
</td>
</tr>
<tr>
<td><code>spark.rpc.retry.wait</code></td>
<td>3s</td>
<td>
How long for an RPC ask operation to wait before starting the next retry.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "before retrying"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duration for an ...

</td>
</tr>
<tr>
<td><code>spark.rpc.askTimeout</code></td>
<td>30s</td>
<td>
How long for an RPC ask operation to wait before timing out.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duration ...

</td>
</tr>
<tr>
<td><code>spark.rpc.lookupTimeout</code></td>
<td>30s</td>
How long for an RPC remote endpoint lookup operation to wait before timing out.
<td>
</td>
</tr>
</table>

#### Scheduling
Expand Down