Fix dependency install in CI

[abidwael]

A collection of CI fixes/improvements:

• For torch nightly tests, pin to torch==2.1.0.dev20230418+cpu. 2.0.0.dev20230213+cpu is out of commission.
• Remove pip install protobuf==3.20.1 workaround for Unit tests fail due to protobuf error: TypeError: Descriptors cannot not be created directly databrickslabs/dbx#257
• Remove horovod-related env variables.

  env:
    HOROVOD_WITH_PYTORCH: 1
    HOROVOD_WITHOUT_MPI: 1
    HOROVOD_WITHOUT_TENSORFLOW: 1
    HOROVOD_WITHOUT_MXNET: 1
  

• Remove --no-use-pep517 and remove ConfigSpace custom installation workaround for Problem when importing ConfigSpace (0.4.17) automl/ConfigSpace#173
• Remove dragonfly-opt from requirements, which causes a numpy installation failure (https://github.com/ludwig-ai/ludwig/actions/runs/4737879639/jobs/8411146481)

[github-actions]

Unit Test Results

    6 files  +    6      6 suites  +6   2h 14m 14s ⏱️ + 2h 14m 14s
169 tests +169  151 ✔️ +151  18 💤 +18  0 ❌ ±0 
214 runs  +214  189 ✔️ +189  25 💤 +25  0 ❌ ±0 

Results for commit 3454f01. ± Comparison against base commit 459e1f1.

♻️ This comment has been updated with latest results.