[CORE] Improve rewrite spark plan rules framework

[ulysses-you]

What changes were proposed in this pull request?

This pr adds a framework RewriteSparkPlanRulesManager to help rewrite spark plan rules and tag transform hint. It moves AddTransformHintRule before rewrite rules. RewriteSparkPlanRulesManager would rewrite plan and apply AddTransformHintRule for each opeartor. If the rewritten opeartor is still TRANSFORM_UNSUPPORTED, using original opeartor.

This pr also removes applyForValidation in AddTransformHintRule, so that we do not need to apply rewrite rules twice.

How was this patch tested?

Pass CI

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[github-actions]

Run Gluten Clickhouse CI

[ulysses-you]

/Benchmark Velox

[github-actions]

Run Gluten Clickhouse CI

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_4645_time.csv	log/native_master_02_04_2024_d1b29e1bc_time.csv	difference	percentage
q1	32.89	33.68	0.792	102.41%
q2	27.51	24.23	-3.282	88.07%
q3	37.30	37.37	0.076	100.20%
q4	39.66	35.96	-3.694	90.69%
q5	70.45	70.39	-0.063	99.91%
q6	5.29	6.99	1.700	132.10%
q7	83.06	84.51	1.449	101.74%
q8	85.17	84.14	-1.027	98.79%
q9	126.84	121.28	-5.555	95.62%
q10	42.23	43.47	1.243	102.94%
q11	20.17	20.46	0.296	101.47%
q12	25.56	26.23	0.674	102.64%
q13	43.97	45.25	1.278	102.91%
q14	16.55	15.93	-0.620	96.25%
q15	29.01	26.74	-2.271	92.17%
q16	13.92	14.11	0.190	101.37%
q17	101.02	102.75	1.734	101.72%
q18	149.37	148.66	-0.718	99.52%
q19	13.83	13.50	-0.333	97.59%
q20	30.23	26.71	-3.518	88.36%
q21	221.38	221.36	-0.020	99.99%
q22	15.27	13.64	-1.635	89.30%
total	1230.67	1217.37	-13.305	98.92%

[ulysses-you]

/Benchmark Velox

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_4645_time.csv	log/native_master_02_04_2024_d1b29e1bc_time.csv	difference	percentage
q1	32.07	33.68	1.616	105.04%
q2	24.20	24.23	0.034	100.14%
q3	36.91	37.37	0.468	101.27%
q4	39.10	35.96	-3.136	91.98%
q5	68.36	70.39	2.028	102.97%
q6	6.97	6.99	0.027	100.38%
q7	83.87	84.51	0.643	100.77%
q8	86.12	84.14	-1.980	97.70%
q9	118.92	121.28	2.367	101.99%
q10	42.54	43.47	0.933	102.19%
q11	21.85	20.46	-1.386	93.66%
q12	27.80	26.23	-1.571	94.35%
q13	45.31	45.25	-0.066	99.85%
q14	21.76	15.93	-5.839	73.17%
q15	31.07	26.74	-4.328	86.07%
q16	13.98	14.11	0.127	100.91%
q17	101.78	102.75	0.977	100.96%
q18	147.90	148.66	0.755	100.51%
q19	12.43	13.50	1.069	108.60%
q20	26.37	26.71	0.335	101.27%
q21	224.80	221.36	-3.440	98.47%
q22	13.73	13.64	-0.091	99.34%
total	1227.83	1217.37	-10.459	99.15%

[ulysses-you]

/Benchmark Velox

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_4645_time.csv	log/native_master_02_04_2024_d1b29e1bc_time.csv	difference	percentage
q1	33.10	33.68	0.584	101.76%
q2	26.07	24.23	-1.842	92.94%
q3	37.33	37.37	0.048	100.13%
q4	37.69	35.96	-1.725	95.42%
q5	70.47	70.39	-0.081	99.88%
q6	6.62	6.99	0.375	105.66%
q7	83.76	84.51	0.750	100.90%
q8	84.37	84.14	-0.229	99.73%
q9	123.97	121.28	-2.685	97.83%
q10	43.79	43.47	-0.318	99.27%
q11	20.38	20.46	0.084	100.41%
q12	24.97	26.23	1.259	105.04%
q13	44.84	45.25	0.412	100.92%
q14	15.34	15.93	0.582	103.80%
q15	29.39	26.74	-2.655	90.97%
q16	13.62	14.11	0.491	103.60%
q17	102.54	102.75	0.216	100.21%
q18	149.06	148.66	-0.408	99.73%
q19	12.47	13.50	1.024	108.21%
q20	26.49	26.71	0.220	100.83%
q21	223.22	221.36	-1.858	99.17%
q22	15.19	13.64	-1.549	89.80%
total	1224.68	1217.37	-7.307	99.40%

[ulysses-you]

cc @liujiayi771 thank you

[liujiayi771]

@ulysses-you Great design. LGTM. Thanks.

[github-actions]

Run Gluten Clickhouse CI

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_4645_time.csv	log/native_master_02_04_2024_d1b29e1bc_time.csv	difference	percentage
q1	34.27	33.68	-0.589	98.28%
q2	24.35	24.23	-0.124	99.49%
q3	38.20	37.37	-0.822	97.85%
q4	37.31	35.96	-1.347	96.39%
q5	70.30	70.39	0.091	100.13%
q6	5.84	6.99	1.150	119.68%
q7	84.80	84.51	-0.292	99.66%
q8	85.45	84.14	-1.310	98.47%
q9	123.43	121.28	-2.148	98.26%
q10	44.81	43.47	-1.340	97.01%
q11	20.06	20.46	0.407	102.03%
q12	29.52	26.23	-3.286	88.87%
q13	45.07	45.25	0.173	100.38%
q14	16.87	15.93	-0.941	94.42%
q15	28.83	26.74	-2.089	92.75%
q16	14.63	14.11	-0.515	96.48%
q17	100.57	102.75	2.181	102.17%
q18	148.77	148.66	-0.111	99.93%
q19	13.47	13.50	0.027	100.20%
q20	26.71	26.71	-0.004	99.98%
q21	223.02	221.36	-1.652	99.26%
q22	14.97	13.64	-1.335	91.08%
total	1231.25	1217.37	-13.877	98.87%