[KYUUBI #307]GetCatalogs supports DSv2 and keeps its backward compatibility

[yaooqinn]

❨?❩

Please add issue ID here?

Fixes #307

Why are the changes needed?

GetCatalogs supports DSv2 and keeps its backward compatibility

Test Plan:

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate
  

• Run test locally before make a pull request

[yaooqinn]

cc @ulysses-you @turboFei @pan3793

[turboFei]

Is it possible that create a util class for each in-compatible method with different spark version.

For example:

spark-3.0

<spark.version>3.0.0</spark.version>
<src.version>3.0</src.version>

source code path: /v3.0/src/main

        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <executions>
                <execution>
                    <id>add-sources</id>
                    <phase>generate-sources</phase>
                    <goals>
                        <goal>add-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>v${src.version}/src/main/scala</source>
                            <source>v${src.version}/src/main/java</source>
                        </sources>
                    </configuration>
                </execution>
                <execution>
                    <id>add-scala-test-sources</id>
                    <phase>generate-test-sources</phase>
                    <goals>
                        <goal>add-test-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>v${src.version}/src/test/scala</source>
                            <source>v${src.version}/src/test/java</source>
                        </sources>
                    </configuration>
                </execution>
            </executions>
        </plugin>

[codecov]

Codecov Report

Merging #307 (59c9d0c) into master (05c64fe) will decrease coverage by 0.11%.
The diff coverage is 72.72%.

@@            Coverage Diff             @@
##           master     #307      +/-   ##
==========================================
- Coverage   79.62%   79.51%   -0.12%     
==========================================
  Files          95       98       +3     
  Lines        3402     3432      +30     
  Branches      399      404       +5     
==========================================
+ Hits         2709     2729      +20     
- Misses        493      497       +4     
- Partials      200      206       +6     

Impacted Files	Coverage Δ
...he/kyuubi/engine/spark/operation/GetCatalogs.scala	83.33% <50.00%> (-16.67%)	⬇️
...rg/apache/kyuubi/engine/spark/shim/Shim_v2_4.scala	50.00% <50.00%> (ø)
...rg/apache/kyuubi/engine/spark/shim/SparkShim.scala	66.66% <66.66%> (ø)
.../kyuubi/engine/spark/operation/GetTableTypes.scala	100.00% <100.00%> (ø)
...rg/apache/kyuubi/engine/spark/shim/Shim_v3_0.scala	100.00% <100.00%> (ø)
...org/apache/kyuubi/ha/server/EmbeddedZkServer.scala	90.00% <0.00%> (-5.00%)	⬇️

[pan3793]

Is it possible that create a util class for each in-compatible method with different spark version.

I recommend introduce a SparkShim rather than create different source folders.

[yaooqinn]

Is it possible that create a util class for each in-compatible method with different spark version.

I recommend introduce a SparkShim rather than create different source folders.

I prefer this way

[github-actions]

Scan Summary

Tool	Critical	High	Medium	Low	Status
Dependency Scan (java)	0	2	5	0	✅
Security Audit for Infrastructure	0	0	0	0	✅
Class File Analyzer	0	6	0	0	❌
Scala Security Audit	0	6	0	0	❌
Shell Script Analysis	0	0	0	0	✅
Java Source Analyzer	0	0	0	1	✅

Recommendation

Please review the findings from Code scanning alerts before approving this pull request. You can also configure the build rules or add suppressions to customize this bot 👍

[yaooqinn]

I add a screenshot in the PR desc, is that what you expected @pan3793

[yaooqinn]

Does Spark v2.4 have the concept of catalog? Maybe here we shall return "", anyway we can check this later when supporting Spark v2.4

[github-actions]

Scan Summary

Tool	Critical	High	Medium	Low	Status
Dependency Scan (java)	0	2	5	0	✅
Shell Script Analysis	0	0	0	0	✅
Class File Analyzer	0	6	0	0	❌
Security Audit for Infrastructure	0	0	0	0	✅
Java Source Analyzer	0	0	0	1	✅
Scala Security Audit	0	6	0	0	❌

Recommendation

Please review the findings from Code scanning alerts before approving this pull request. You can also configure the build rules or add suppressions to customize this bot 👍

[pan3793]

I add a screenshot in the PR desc, is that what you expected @pan3793

Nice! LGTM, just few comments.