Spark tests fail with AttributeError: 'pyarrow.lib.Schema' object has no attribute 'to_arrow_schema'

[maxhgerlach]

Spark tests fail with

Traceback (most recent call last):
  File "/horovod/examples/spark/pytorch/pytorch_spark_mnist.py", line 122, in <module>
    torch_model = torch_estimator.fit(train_df).setOutputCols(['label_prob'])
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/estimator.py", line 35, in fit
    return super(HorovodEstimator, self).fit(df, params)
  File "/usr/local/lib/python3.8/dist-packages/pyspark/ml/base.py", line 205, in fit
    return self._fit(dataset)
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/estimator.py", line 68, in _fit
    with util.prepare_data(backend.num_processes(),
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/util.py", line 735, in prepare_data
    dataset_idx = _get_or_create_dataset(key, store, df, feature_columns, label_columns,
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/util.py", line 672, in _get_or_create_dataset
    train_rows, val_rows, pq_metadata, avg_row_size = get_simple_meta_from_parquet(
  File "/usr/local/lib/python3.8/dist-packages/horovod/spark/common/util.py", line 495, in get_simple_meta_from_parquet
    train_data_schema = train_data.schema.to_arrow_schema()
AttributeError: 'pyarrow.lib.Schema' object has no attribute 'to_arrow_schema'

https://github.com/horovod/horovod/actions/runs/4025636527/jobs/6919132641

It's probably related to the 11.0 release of pyarrow. https://pypi.org/project/pyarrow/#history