[BUG] TiSpark recent release is slower than before

[birdstorm]

The problem is located that in the recent release of TiSpark, the default value of PARTITION_PER_SPLIT is changed from 10 to 1. It results in increasing Spark tasks.

some related problems:

• ScanRequest receive a slower response from tikv when scanning meta data
  • cause: scanning is not concurrent.
• Memory usage incresed.
  • cause: the memory usage of ColumnVector should be optimized.

Affected versions: v2.3.14 to v2.3.16, v2.4.1

[crabo]

"Memory usage": Pls also check DAGIterator.process(), the underlying grpc always thow OutOfDirectMemoryError even in unpooled mode. As 10Million rows table scan in ETL, Off-Heap mem is requried roughly 5GB, that's really a big waste.