[#630] feat(client): Disable the localShuffleReader by default in Spa…

…rk3 client (#636)

### What changes were proposed in this pull request?

Disable the localShuffleReader by default in Spark3 client

### Why are the changes needed?

in `RssShuffleManager`, the local shuffle reader should be disabled by default (`spark.sql.adaptive.localShuffleReader.enabled=false`), as it would cause too many random small IOs and network connections with shuffle servers.

### Does this PR introduce _any_ user-facing change?

Yes

### How was this patch tested?

Don't need

client-spark/spark3/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java

@@ -201,6 +201,8 @@ public RssShuffleManager(SparkConf conf, boolean isDriver) {
     // External shuffle service is not supported when using remote shuffle service
     sparkConf.set("spark.shuffle.service.enabled", "false");
     LOG.info("Disable external shuffle service in RssShuffleManager.");
+    sparkConf.set("spark.sql.adaptive.localShuffleReader.enabled", "false");
+    LOG.info("Disable local shuffle reader in RssShuffleManager.");
     taskToSuccessBlockIds = Maps.newConcurrentMap();
     taskToFailedBlockIds = Maps.newConcurrentMap();
     // for non-driver executor, start a thread for sending shuffle data to shuffle server

docs/client_guide.md

@@ -80,6 +80,11 @@ and Continuous partition assignment mechanism.
         # Default value is 1.0, used to estimate task concurrency, how likely is this part of the resource between spark.dynamicAllocation.minExecutors and spark.dynamicAllocation.maxExecutors to be allocated
         --conf spark.rss.estimate.task.concurrency.dynamic.factor=1.0
         ```
+
+Since v0.8.0, `RssShuffleManager` would disable local shuffle reader(`set spark.sql.adaptive.localShuffleReader.enabled=false`) optimization by default.
+
+Local shuffle reader as its name indicates is suitable and optimized for spark's external shuffle service, and shall not be used for remote shuffle service. It would cause many random small IOs and network connections with Uniffle's shuffle server
+
 ### Deploy MapReduce Client Plugin
 
 1. Add client jar to the classpath of each NodeManager, e.g., <HADOOP>/share/hadoop/mapreduce/