Summary:
Add functional test reporting when running tests via spark clusters.
This is skipped if the csi environment variables are not set.

Allow explicit test list to be passed in for use in spark-testing.
This enables autoamtion to query test failures from previous run and specify them exactly instead of
using regex filters.

run_tests_on_spark.py script has three (related) improvements:
* When dynamically generating test list from C++ binaries, just collect them locally without a
  spark job. This avoids the spark overhead and can be done in a short time (~90 second) that would
  normally be idle waiting for spark resources.
* A new ignore_list option that can be used to remove tests from the generated test list. This can
  be used when re-running a test job that did not complete and the missing tests are unknown. By
  passing the list of tests already run & passed, the new run can complete the job efficently.
* A new disable_list option that can be used to remove tests from the generated test list, and also
  report them as skipped. This can be used to provide a list of disabled (aka muted) tests that need
  not be run.

Original commit: 9fb769e4b1fc02eac40892bf9ebb2fafc277fe01 / D46553
                 25c7d45f4f5c8b12c70b1a0156d2935615128220 / D46018
                 b487d74ffb4dc7ac526c2dc44f6817abcb77b22a / D45899
                 5de9b9f22d728521db944d07ef0e970189b1ae12 / D45643
                 f354b56196839db3e4e068720f72badf8731b7cd / D44934
                 d8017672bb953353ead5f4f2e4597081e6566119 / D44779
                 03de75584155ff143f9bcbd0a280932455a528cf / D44407
                 453eb9a115f4964e437092d00b74a7c29e93a788 / PR #27406
                 6d09afd58048da6974b20381cb34633fe2086ee7 / D43562

Test Plan: Jenkins

Reviewers: jharveysmith

Reviewed By: jharveysmith

Subscribers: devops

Differential Revision: https://phorge.dev.yugabyte.com/D47301