[tests] enable benchmark unit tests on XPU

[faaany]

What does this PR do?

Test Summary before Fix:

================================================== short test summary info ===================================================
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:149: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:38: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:70: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:54: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:165: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:85: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:102: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:182: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:202: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:118: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:133: test requires TensorFlow
FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_fp16 - ValueError: Mixed precision is possible only for GPU.
FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_train_no_configs_fp16 - ValueError: Mixed precision is possible only for GPU.

Test Summary after Fix:

$ RUN_SLOW=1 TRANSFORMERS_TEST_DEVICE="xpu" TRANSFORMERS_TEST_DEVICE_SPEC="spec.py" python -m pytest -ra tests/benchmark
==================================================== test session starts =====================================================
platform linux -- Python 3.10.13, pytest-8.0.0, pluggy-1.4.0
rootdir: /soft/fanli/transformers
configfile: pyproject.toml
plugins: xdist-3.5.0, anyio-4.2.0, hypothesis-6.98.0
collected 24 items                                                                                                           

tests/benchmark/test_benchmark.py .............                                                                        [ 54%]
tests/benchmark/test_benchmark_tf.py sssssssssss                                                                       [100%]
================================================== short test summary info ===================================================
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:149: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:38: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:70: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:54: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:165: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:85: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:102: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:182: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:202: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:118: test requires TensorFlow
SKIPPED [1] tests/benchmark/test_benchmark_tf.py:133: test requires TensorFlow
======================================== 13 passed, 11 skipped, 30 warnings in 10.63s ========================================

[faaany]

@amyeroberts @muellerz pls have a review, thx!

[amyeroberts]

LGTM - thanks for adding!