[SPARK-37649][PYTHON] Switch default index to distributed-sequence by default in pandas API on Spark

[HyukjinKwon]

What changes were proposed in this pull request?

This PR proposes to switch default index to distributed-sequence by default.

Why are the changes needed?

sequence type relies on sending all data to one executor that easily causes OOM and make the computation slow
We should better switch to distributed-sequence type that truly distributes the data.

In my own internal benchmark, it boosts the performance around 2 ~ 3 times faster on average.

Does this PR introduce any user-facing change?

Ideally no. Order might be affected but that's not already guaranteed.

How was this patch tested?

Existing CI should test it out.

[HyukjinKwon]

cc @ueshin @xinrong-databricks @itholic

[ueshin]

There are some places mentioning the default type in the docs, like:

spark/python/docs/source/user_guide/pandas_on_spark/best_practices.rst

Line 238 in 3704d4d

This default index is ``sequence`` which requires the computation on single partition which is discouraged. If you plan

spark/python/docs/source/user_guide/pandas_on_spark/options.rst

Line 163 in 3704d4d

This index type should be avoided when the data is large. This is default. See the example below:

[SparkQA]

Test build #146202 has finished for PR 34902 at commit 3704d4d.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[itholic]

LGTM

[ueshin]

LGTM, pending tests.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50676/

[SparkQA]

Test build #146204 has finished for PR 34902 at commit 969911a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50678/

[HyukjinKwon]

All tests passed at https://github.com/HyukjinKwon/spark/actions/runs/1580641441.

Merged to master.

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50676/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50678/