build: make databricks tests use instance pools to remove state management

[mhamilton723]

No description provided.

[mhamilton723]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[codecov]

Codecov Report

Merging #673 into master will decrease coverage by 29.29%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #673      +/-   ##
==========================================
- Coverage    79.9%   50.61%   -29.3%     
==========================================
  Files         225      225              
  Lines        9033     9033              
  Branches      483      483              
==========================================
- Hits         7218     4572    -2646     
- Misses       1815     4461    +2646

Impacted Files	Coverage Δ
.../com/microsoft/ml/spark/lightgbm/TrainParams.scala	0% <0%> (-100%)	⬇️
...cala/com/microsoft/ml/spark/io/binary/Binary.scala	0% <0%> (-100%)	⬇️
.../microsoft/ml/spark/lightgbm/LightGBMBooster.scala	0% <0%> (-98.77%)	⬇️
...ql/execution/streaming/continuous/HTTPSinkV2.scala	0% <0%> (-96.56%)	⬇️
.../microsoft/ml/spark/lightgbm/LightGBMDataset.scala	0% <0%> (-96.16%)	⬇️
.../execution/streaming/continuous/HTTPSourceV2.scala	0% <0%> (-92.6%)	⬇️
...a/com/microsoft/ml/spark/lightgbm/TrainUtils.scala	0% <0%> (-90.18%)	⬇️
...om/microsoft/ml/spark/lightgbm/LightGBMUtils.scala	7.05% <0%> (-89.42%)	⬇️
...che/spark/sql/execution/streaming/HTTPSource.scala	0% <0%> (-87.81%)	⬇️
...la/com/microsoft/ml/spark/io/http/HTTPSchema.scala	2.87% <0%> (-85.62%)	⬇️
... and 60 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9805996...c51a61f. Read the comment docs.

[mhamilton723]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).