[data] add hive catalog

[Jay-ju]

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[gvspraveen]

@Bye-legumes Thanks for the contribution. Can you please fix the test failures?

[2025-03-28T07:24:35Z] python/ray/data/tests/test_hive_catalog.py::test_file_formats[PARQUET-parquet] -- Test timed out at 2025-03-28 07:22:34 UTC --
--
  | [2025-03-28T07:24:35Z] ================================================================================
  | [2025-03-28T07:24:35Z] ==================== Test output for //python/ray/data:test_hive_catalog:
  | [2025-03-28T07:24:35Z] ============================= test session starts ==============================
  | [2025-03-28T07:24:35Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniconda/bin/python3
  | [2025-03-28T07:24:35Z] cachedir: .pytest_cache
  | [2025-03-28T07:24:35Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/com_github_ray_project_ray
  | [2025-03-28T07:24:35Z] configfile: pytest.ini
  | [2025-03-28T07:24:35Z] plugins: mock-3.14.0, repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, httpserver-1.0.6, lazy-fixture-0.6.3, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3
  | [2025-03-28T07:24:35Z] asyncio: mode=Mode.AUTO
  | [2025-03-28T07:24:35Z] timeout: 180.0s
  | [2025-03-28T07:24:35Z] timeout method: signal
  | [2025-03-28T07:24:35Z] timeout func_only: False
  | [2025-03-28T07:24:35Z] collecting ... collected 5 items
  | [2025-03-28T07:24:35Z]
  | [2025-03-28T07:24:35Z] python/ray/data/tests/test_hive_catalog.py::test_file_formats[PARQUET-parquet] -- Test timed out at 2025-03-28 07:23:34 UTC --
  | [2025-03-28T07:24:35Z] ================================================================================
  | [2025-03-28T07:24:35Z] ==================== Test output for //python/ray/data:test_hive_catalog:
  | [2025-03-28T07:24:35Z] ============================= test session starts ==============================
  | [2025-03-28T07:24:35Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniconda/bin/python3
  | [2025-03-28T07:24:35Z] cachedir: .pytest_cache
  | [2025-03-28T07:24:35Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/com_github_ray_project_ray
  | [2025-03-28T07:24:35Z] configfile: pytest.ini
  | [2025-03-28T07:24:35Z] plugins: mock-3.14.0, repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, httpserver-1.0.6, lazy-fixture-0.6.3, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3
  | [2025-03-28T07:24:35Z] asyncio: mode=Mode.AUTO
  | [2025-03-28T07:24:35Z] timeout: 180.0s
  | [2025-03-28T07:24:35Z] timeout method: signal
  | [2025-03-28T07:24:35Z] timeout func_only: False
  | [2025-03-28T07:24:35Z] collecting ... collected 5 items
  | [2025-03-28T07:24:35Z]
  | [2025-03-28T07:24:35Z] python/ray/data/tests/test_hive_catalog.py::test_file_formats[PARQUET-parquet] -- Test timed out at 2025-03-28 07:24:34 UTC --

[Jay-ju]

@Bye-legumes Thanks for the contribution. Can you please fix the test failures?

[2025-03-28T07:24:35Z] python/ray/data/tests/test_hive_catalog.py::test_file_formats[PARQUET-parquet] -- Test timed out at 2025-03-28 07:22:34 UTC --
--
  | [2025-03-28T07:24:35Z] ================================================================================
  | [2025-03-28T07:24:35Z] ==================== Test output for //python/ray/data:test_hive_catalog:
  | [2025-03-28T07:24:35Z] ============================= test session starts ==============================
  | [2025-03-28T07:24:35Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniconda/bin/python3
  | [2025-03-28T07:24:35Z] cachedir: .pytest_cache
  | [2025-03-28T07:24:35Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/com_github_ray_project_ray
  | [2025-03-28T07:24:35Z] configfile: pytest.ini
  | [2025-03-28T07:24:35Z] plugins: mock-3.14.0, repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, httpserver-1.0.6, lazy-fixture-0.6.3, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3
  | [2025-03-28T07:24:35Z] asyncio: mode=Mode.AUTO
  | [2025-03-28T07:24:35Z] timeout: 180.0s
  | [2025-03-28T07:24:35Z] timeout method: signal
  | [2025-03-28T07:24:35Z] timeout func_only: False
  | [2025-03-28T07:24:35Z] collecting ... collected 5 items
  | [2025-03-28T07:24:35Z]
  | [2025-03-28T07:24:35Z] python/ray/data/tests/test_hive_catalog.py::test_file_formats[PARQUET-parquet] -- Test timed out at 2025-03-28 07:23:34 UTC --
  | [2025-03-28T07:24:35Z] ================================================================================
  | [2025-03-28T07:24:35Z] ==================== Test output for //python/ray/data:test_hive_catalog:
  | [2025-03-28T07:24:35Z] ============================= test session starts ==============================
  | [2025-03-28T07:24:35Z] platform linux -- Python 3.12.9, pytest-7.4.4, pluggy-1.3.0 -- /opt/miniconda/bin/python3
  | [2025-03-28T07:24:35Z] cachedir: .pytest_cache
  | [2025-03-28T07:24:35Z] rootdir: /root/.cache/bazel/_bazel_root/1df605deb6d24fc8068f6e25793ec703/execroot/com_github_ray_project_ray
  | [2025-03-28T07:24:35Z] configfile: pytest.ini
  | [2025-03-28T07:24:35Z] plugins: mock-3.14.0, repeat-0.9.3, anyio-3.7.1, fugue-0.8.7, aiohttp-1.1.0, asyncio-0.17.2, docker-tools-3.1.3, forked-1.4.0, httpserver-1.0.6, lazy-fixture-0.6.3, remotedata-0.3.2, rerunfailures-11.1.2, sphinx-0.5.1.dev0, sugar-0.9.5, timeout-2.1.0, typeguard-2.13.3
  | [2025-03-28T07:24:35Z] asyncio: mode=Mode.AUTO
  | [2025-03-28T07:24:35Z] timeout: 180.0s
  | [2025-03-28T07:24:35Z] timeout method: signal
  | [2025-03-28T07:24:35Z] timeout func_only: False
  | [2025-03-28T07:24:35Z] collecting ... collected 5 items
  | [2025-03-28T07:24:35Z]
  | [2025-03-28T07:24:35Z] python/ray/data/tests/test_hive_catalog.py::test_file_formats[PARQUET-parquet] -- Test timed out at 2025-03-28 07:24:34 UTC --

@gvspraveen The address tested here is my local address. There is no problem with local testing. In the test environment, due to the absence of the HMS server, a timeout error will be reported. However, I have added a check on whether the link is accessible in the UT to determine whether to run the test cases.

[github-actions]

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[github-actions]

This pull request has been automatically closed because there has been no more activity in the 14 days
since being marked stale.

Please feel free to reopen or open a new pull request if you'd still like this to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for your contribution!