[KYUUBI #6281][PY] Initialize github action for python unit testing

[sudohainguyen]

🔍 Description

Issue References 🔗

This pull request fixes #6281

Describe Your Solution 🔧

The change initialize a CI job to run unit testing on python client, including:

• Set up Github Action based on docker-compose
• Update test cases and test succeeded for dialect presto and trino
• Temporary disabled hive related test due to test cases are not valid, not about connection
• Update dev dependencies to support python 3.10
• Speed up testing with pytest-xdist plugin

Types of changes 🔖

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request ⚰️

Not able to ran unit test in local and on CI

Behavior With This Pull Request 🎉

Able to run and partially cover a couple of test cases

Related Unit Tests

No

Additional notes

Next action is about fixing failing test cases or considering skipping some of them if necessary

---

Checklist 📝

• This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[sudohainguyen]

@pan3793 not sure this is which type of change, new feature or bug fix?

[yaooqinn]

hoverkraft-tech/compose-action@v1.5.1 is not allowed to be used in apache/kyuubi. Actions in this workflow must be: within a repository owned by apache, created by GitHub, verified in the GitHub Marketplace, or matching the following: /@[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9]+, AdoptOpenJDK/install-jdk@, JamesIves/github-pages-deploy-action@5dc1d5a, TobKed/label-when-approved-action@, actions-cool/issues-helper@, actions-rs/, al-cheb/configure-pagefile-action@, amannn/action-semantic-pull-request@, apache/, burrunan/gradle-cache-action@, bytedeco/javacpp-presets/.github/actions/, chromaui/action@, codecov/codecov-action@, conda-incubator/setup-miniconda@, container-tools/kind-action@, container-tools/microshift-action@, dawidd6/action-download-artifact@, delaguardo/setup-graalvm@, docker://jekyll/jekyll:, docker://pandoc/core:2.9, eps1lon/actions-label-merge-conflict@, gaurav-nelson/github-action-markdown-link-che...

Are there any other options that we can consider?

[sudohainguyen]

thanks for informing, where can I see that message in full version? @yaooqinn

[yaooqinn]

https://github.com/apache/kyuubi/actions/runs/8868934136

You can navi to this page via Checks tab of this PR

[sudohainguyen]

ah thanks a lot, that's why it doesn't appear below

[yaooqinn]

Yeah, We can refer to other Apache repos for available actions

[sudohainguyen]

@pan3793 also suggested me another adopting this approach
I went for docker-compose first because it provides isolation and flexibility with less set up efforts

let me try this one

[pan3793]

docker compose is fine too. and trino provides some pre-built images for Hadoop/Hive/Spark https://github.com/trinodb/docker-images

[pan3793]

@sudohainguyen hadoop-testing is for local dev. for CI case, docker compose / single docker image approach is preferred. so I suggest u refer to Trino's approach. Apache has strict rules for GitHub Actions plugins, you may need to install docker compose manually or leverage testcontainers(we already use the Java/Scala one)

[sudohainguyen]

just switched action provider to docker and seems working now 🙌

failed somewhere, testing on my personal repo

[sudohainguyen]

ok works now @pan3793
update:

• Update test cases and test succeeded for dialect presto and trino
• Temporary disable hive related test due to test cases are not valid, not about connection, will resolve in the next PR
• Speed up testing with pytest-xdist plugin

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 58.44%. Comparing base (ef28a61) to head (96ef831).
Report is 1 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #6343      +/-   ##
============================================
- Coverage     58.47%   58.44%   -0.04%     
  Complexity       24       24              
============================================
  Files           653      653              
  Lines         39880    39880              
  Branches       5481     5481              
============================================
- Hits          23320    23306      -14     
- Misses        14071    14075       +4     
- Partials       2489     2499      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sudohainguyen]

@pan3793 could we process this MR please 👀 ?

[pan3793]

Temporary disabled hive related test due to test cases are not valid

Can you elaborate more on it?

[pan3793]

hadoop 2.0.0 is extremely old, why choose this image?

[sudohainguyen]

was using existing images on the internet to save overhead preparation time 😄

[sudohainguyen]

do you think in the future this should be kyuubi image instead?

[pan3793]

@sudohainguyen sorry for being late, just got back to the office today

[sudohainguyen]

Temporary disabled hive related test due to test cases are not valid

Can you elaborate more on it?

yep, I've ran all test cases and can see the result here
there are 2 failed cases but I'm not sure if we should skip them or not, think we can work this out in the next PR?

[pan3793]

there are 2 failed cases

but I found 5 @pytest.mark.skip in changes, and one is applied to the class, is it expected?

[sudohainguyen]

yes this relates to hive test cases we need to resolve in the upcoming PR 😄,

annotation parts are to skip those test cases trigger restarted hive server, we cannot replicate those ones

[pan3793]

Thanks, merged to master