[Data] Fix test_huggingface for arrow nightly (#42341)

For Arrow nightly Data CI tests, the Pyarrow version is of the form: 15.0.0.dev404, which is not able to be handled by the previous datasets.Version utility we were using to gate the test. Use generic Version class instead, which can handle these version types.

Signed-off-by: Scott Lee <sjl@anyscale.com>

python/ray/data/tests/test_huggingface.py

@@ -1,6 +1,7 @@
 import datasets
 import pyarrow
 import pytest
+from packaging.version import Version
 
 import ray
 from ray.tests.conftest import *  # noqa
@@ -47,7 +48,7 @@ def test_from_huggingface(ray_start_regular_shared):
     reason="IterableDataset.iter() added in 2.8.0",
 )
 @pytest.mark.skipif(
-    datasets.Version(pyarrow.__version__) < datasets.Version("8.0.0"),
+    Version(pyarrow.__version__) < Version("8.0.0"),
     reason="pyarrow.Table.to_reader() added in 8.0.0",
 )
 # Note, pandas is excluded here because IterableDatasets do not support pandas format.