Add a job for ObjC bazel tests

[jtattermusch]

Progress on grpc/grpc-ios#74 and related issues.

Create a test job that runs bazel test for targets under //src/objective-c/ that currently seem to be buildable.

• having a working bazel test for stuff under //src/objective-c is a good starting point for more bazelification of objC
• it makes sure that at least some objC stuff is buildable and runnable under bazel
• it makes sense to build more targets in a single job, since bazel will only build everything that's needed once and there's a significant overlap
• since running objC tests requires having an interop_server to be running locally, the script first builds interop_server locally (and uses remote cache to speed that up) and then runs it on background

Future work (ideally not done by me, others can pick it up from here)

• run more targets with bazel (lots of objC test tasks from run_tests.py

  grpc/tools/run_tests/run_tests.py

  Lines 984 to 1122 in 1d94aa9

  	out.append(
  	self.config.job_spec(
  	['src/objective-c/tests/build_one_example_bazel.sh'],
  	timeout_seconds=10 * 60,
  	shortname='ios-buildtest-example-sample',
  	cpu_cost=1e6,
  	environ={
  	'SCHEME': 'Sample',
  	'EXAMPLE_PATH': 'src/objective-c/examples/Sample',
  	'FRAMEWORKS': 'NO'
  	}))
  	# Currently not supporting compiling as frameworks in Bazel
  	out.append(
  	self.config.job_spec(
  	['src/objective-c/tests/build_one_example.sh'],
  	timeout_seconds=20 * 60,
  	shortname='ios-buildtest-example-sample-frameworks',
  	cpu_cost=1e6,
  	environ={
  	'SCHEME': 'Sample',
  	'EXAMPLE_PATH': 'src/objective-c/examples/Sample',
  	'FRAMEWORKS': 'YES'
  	}))
  	out.append(
  	self.config.job_spec(
  	['src/objective-c/tests/build_one_example.sh'],
  	timeout_seconds=20 * 60,
  	shortname='ios-buildtest-example-switftsample',
  	cpu_cost=1e6,
  	environ={
  	'SCHEME': 'SwiftSample',
  	'EXAMPLE_PATH': 'src/objective-c/examples/SwiftSample'
  	}))
  	out.append(
  	self.config.job_spec(
  	['src/objective-c/tests/build_one_example_bazel.sh'],
  	timeout_seconds=10 * 60,
  	shortname='ios-buildtest-example-tvOS-sample',
  	cpu_cost=1e6,
  	environ={
  	'SCHEME': 'tvOS-sample',
  	'EXAMPLE_PATH': 'src/objective-c/examples/tvOS-sample',
  	'FRAMEWORKS': 'NO'
  	}))
  	# Disabled due to #20258
  	# TODO (mxyan): Reenable this test when #20258 is resolved.
  	# out.append(
  	# self.config.job_spec(
  	# ['src/objective-c/tests/build_one_example_bazel.sh'],
  	# timeout_seconds=20 * 60,
  	# shortname='ios-buildtest-example-watchOS-sample',
  	# cpu_cost=1e6,
  	# environ={
  	# 'SCHEME': 'watchOS-sample-WatchKit-App',
  	# 'EXAMPLE_PATH': 'src/objective-c/examples/watchOS-sample',
  	# 'FRAMEWORKS': 'NO'
  	# }))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_plugin_tests.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-plugintest',
  	cpu_cost=1e6,
  	environ=_FORCE_ENVIRON_FOR_WRAPPERS))
  	out.append(
  	self.config.job_spec(
  	['src/objective-c/tests/run_plugin_option_tests.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-plugin-option-test',
  	cpu_cost=1e6,
  	environ=_FORCE_ENVIRON_FOR_WRAPPERS))
  	out.append(
  	self.config.job_spec(
  	['test/core/iomgr/ios/CFStreamTests/build_and_run_tests.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-cfstream-tests',
  	cpu_cost=1e6,
  	environ=_FORCE_ENVIRON_FOR_WRAPPERS))
  	out.append(
  	self.config.job_spec(
  	['src/objective-c/tests/CoreTests/build_and_run_tests.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-core-tests',
  	cpu_cost=1e6,
  	environ=_FORCE_ENVIRON_FOR_WRAPPERS))
  	# TODO: replace with run_one_test_bazel.sh when Bazel-Xcode is stable
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-unittests',
  	cpu_cost=1e6,
  	environ={'SCHEME': 'UnitTests'}))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-interoptests',
  	cpu_cost=1e6,
  	environ={'SCHEME': 'InteropTests'}))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-test-cronettests',
  	cpu_cost=1e6,
  	environ={'SCHEME': 'CronetTests'}))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=30 * 60,
  	shortname='ios-perf-test',
  	cpu_cost=1e6,
  	environ={'SCHEME': 'PerfTests'}))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=30 * 60,
  	shortname='ios-perf-test-posix',
  	cpu_cost=1e6,
  	environ={'SCHEME': 'PerfTestsPosix'}))
  	out.append(
  	self.config.job_spec(['test/cpp/ios/build_and_run_tests.sh'],
  	timeout_seconds=60 * 60,
  	shortname='ios-cpp-test-cronet',
  	cpu_cost=1e6,
  	environ=_FORCE_ENVIRON_FOR_WRAPPERS))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=60 * 60,
  	shortname='mac-test-basictests',
  	cpu_cost=1e6,
  	environ={
  	'SCHEME': 'MacTests',
  	'PLATFORM': 'macos'
  	}))
  	out.append(
  	self.config.job_spec(['src/objective-c/tests/run_one_test.sh'],
  	timeout_seconds=30 * 60,
  	shortname='tvos-test-basictests',
  	cpu_cost=1e6,
  	environ={
  	'SCHEME': 'TvTests',
  	'PLATFORM': 'tvos'
  	}))

  currently lack corresponding bazel targets). Once bazel targets are added/fixed (see TODOs), they can be added under this test job).
• make it easier to run tests locally with bazel as well
  - currently staring the interop server locally is a pain and requires a lot of boilerplate
  - currently running locally is a pain with bazel 4.2.1 since it has a bug in python3 support (bazel 5.1.1 works just fine) and installing python2 locally is a pain recent version of OS X (which is what our laptops have). At the same time we don't want to update the bazel version throughout our repository just because of this. For context on the bug (see Error : env: python: No such file or directory google-ai-edge/mediapipe#3184 and bazel fix bazelbuild/bazel@2945ef5)
• remove some test tasks from run_tests.py once we have enough coverage for them in bazel.

[jtattermusch]

Adhoc test run is here:
https://source.cloud.google.com/results/invocations/bc289b4d-04a0-4318-a221-d302fee76f2e/targets (and the corresponding bazel invocation: https://source.cloud.google.com/results/invocations/7432695e-c957-42c3-8869-d8160490a18e)

The duration of the job is only 18min, and further speedup is possible by enabling remote caching for the ObjC stuff (but I'd only do it as a followup)

[jtattermusch]

@HannahShiSFB

[dennycd]

should we time these two scripts to get a measurement of how many #sec are spent in the preparation phase ? The prepare_build_macos_rc script currently triggers a number of cocoapod steps which may take up time on a new CI node.

[jtattermusch]

I looked at the scripts and with the current setup, they should be almost instantaneous.
I think the slow steps you're referring to are protected with PREPARE_BUILD_INSTALL_DEPS_OBJC=true,
but that's not being set in this job (looks like those steps are unnecessary for bazel) and those steps are being skipped.

grpc/tools/internal_ci/helper_scripts/prepare_build_macos_rc

Line 97 in d19fa62

if [ "${PREPARE_BUILD_INSTALL_DEPS_OBJC}" == "true" ]

[dennycd]

NIT: couple of optimization we can do here e.g. only build & run interop server if the test requires it (e.g. no need for unit test , or remote test etc.). Will follow up on this part

[jtattermusch]

sg, let's look into this as a followup. My main goal here was to simply get SOME well configured bazel build up and running.

[dennycd]

might be good to reduce this down to e.g. 20 ~ 30min so that it fails fast in a bazel test

[jtattermusch]

we've had problems with timeouts for jobs being set too low in various scenarios (e.g. there's some wait to get a worker provisioned if there a queue etc), so I tend to set timeout_mins with some extra buffer to avoid problems.
Value of 90 is consistent with what we do for other jobs (some of which are also much faster then 90mins), so let's leave as is.
The question of how to set timeout_mins seems orthogonal to what's being setup in this PR and it also requires more though (and in my experience spending too much time trying to figure out the "right" value to use wasn't very productive in the past).

[dennycd]

• • currently running locally is a pain with bazel 4.2.1 since it has a bug in python3 support (bazel 5.1.1 works just fine) and installing python2 locally is a pain recent version of OS X (which is what our laptops have). At the same time we don't want to update the bazel version throughout our repository just because of this. For context on the bug (see Error : env: python: No such file or directory google/mediapipe#3184 and bazel fix bazelbuild/bazel@2945ef5)

Thanks Jan. Yep we noticed this issue with Mac OS X Monterey and bazel 4.2.x. Do we know when we can upgrade to bazel 5.x in gRPC ? Otherwise, we would need to start investigating a workaround since this will be blockage for us to to migrate to bazel iOS test.

[jtattermusch]

• • currently running locally is a pain with bazel 4.2.1 since it has a bug in python3 support (bazel 5.1.1 works just fine) and installing python2 locally is a pain recent version of OS X (which is what our laptops have). At the same time we don't want to update the bazel version throughout our repository just because of this. For context on the bug (see Error : env: python: No such file or directory google/mediapipe#3184 and bazel fix bazelbuild/bazel@2945ef5)

Thanks Jan. Yep we noticed this issue with Mac OS X Monterey and bazel 4.2.x. Do we know when we can upgrade to bazel 5.x in gRPC ? Otherwise, we would need to start investigating a workaround since this will be blockage for us to to migrate to bazel iOS test.

I don't think we should migrate to bazel 5 just because of this. If there are other good reasons to migrate, we can consider doing that, but note that our support policy says that we should support "the two most recent LTS versions of Bazel" (currently that is 4.x and 5.x)
https://github.com/grpc/grpc/blob/master/doc/bazel_support.md#supported-versions
Have you tried the workaround to install python2 locally on your machine?

[jtattermusch]

bazel C/C++ failure is an unrelated flake (since this PR only adds a new script for objC tests and doesn't change the existing build at all)l