upgrade protobuf to 3.18.1 (automatically generated)

[jtattermusch]

Same as #27712, but generated automatically with the release_open_source.py script.

Let's double check the PRs are identical before merging one of them.

[markdroth]

From a quick skim, they look the same to me. I'll leave it up to you as to which one you want to merge.

[jtattermusch]

The python bazel build failure seems as a legitimate failure:
https://source.cloud.google.com/results/invocations/af161a15-1c8c-475c-8cfb-4aea52d101c0/log ?

INFO: From Testing //src/python/grpcio_tests/tests/health_check:health_servicer_test.python2:
==================== Test output for //src/python/grpcio_tests/tests/health_check:health_servicer_test.python2:
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/3139/execroot/com_github_grpc_grpc/bazel-out/k8-py2-fastbuild/bin/src/python/grpcio_tests/tests/health_check/health_servicer_test.python2.runfiles/com_github_grpc_grpc/src/python/grpcio_tests/tests/health_check/_health_servicer_test.py", line 22, in <module>
    from grpc_health.v1 import health
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/3139/execroot/com_github_grpc_grpc/bazel-out/k8-py2-fastbuild/bin/src/python/grpcio_tests/tests/health_check/health_servicer_test.python2.runfiles/com_github_grpc_grpc/src/python/grpcio_health_checking/grpc_health/v1/health.py", line 21, in <module>
    from grpc_health.v1 import health_pb2 as _health_pb2
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/3139/execroot/com_github_grpc_grpc/bazel-out/k8-py2-fastbuild/bin/src/python/grpcio_tests/tests/health_check/health_servicer_test.python2.runfiles/com_github_grpc_grpc/src/python/grpcio_health_checking/grpc_health/v1/health_pb2.py", line 5, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/sandbox/processwrapper-sandbox/3139/execroot/com_github_grpc_grpc/bazel-out/k8-py2-fastbuild/bin/src/python/grpcio_tests/tests/health_check/health_servicer_test.python2.runfiles/com_google_protobuf/python/google/protobuf/descriptor.py", line 113
    class DescriptorBase(metaclass=DescriptorMetaclass):
                                  ^
SyntaxError: invalid syntax
================================================================================
FAIL: //src/python/grpcio_tests/tests/csds:test_csds.python2 (see /root/.cache/bazel/_bazel_root/d2dc70c3d9da3fab488ba0dcbbd35051/execroot/com_github_grpc_grpc/bazel-out/k8-py2-fastbuild/testlogs/src/python/grpcio_tests/tests/csds/test_csds.python2/test.log)

[jtattermusch]

Looks like the python problem is
protocolbuffers/protobuf#9045, and the resolution was that the protobuf 3.18.0 release got yanked. Release 3.18.1 is still available though and it seems to be suffering from the same problem
(invalid syntax in class DescriptorBase(metaclass=DescriptorMetaclass))

@acozzette what is the recommended resolution here? Should we be upgrading python protobuf to 3.18.1 or not?

CC @gnossen @lidizheng

[lidizheng]

I'm in favor of getting rid of all Python 2 tests. The only concern I have is the potential impact we might have for GAE users. I have been running TGP for the removal (cl/353911233) every couple months, last time (3 months ago) there were only one target (GAE) left in google3 that still uses Python 2.

[markdroth]

Sorry for not updating the issue with current state. I spoke with @gnossen about this yesterday, and he said he'd put together a PR to remove the python2 tests.

[markdroth]

I've triggered a re-run of the tests now that #27776 has been merged.

[markdroth]

@dennycd Can you please take a look at the iOS test failure and help us figure out if this is related to the protobuf upgrade, and if so, advise as to how to fix it? Thanks!

[jtattermusch]

The objc iOS failure is a timeout and seems to be happening on master as well:
(e.g. here: https://source.cloud.google.com/results/invocations/912975a9-03ed-4a59-9ead-a43d547830a8/log)

The C++ iOS failure seems similar to b/200609622, but I'm not 100% sure.

[markdroth]

Yeah, I'm aware that there is a general issue going on there, but I'm a little suspicious, since this particular failure is in a protobuf-related file. Let me try rerunning that test and we'll see if it happens again.

[markdroth]

The iOS C++ test is continuing to fail in exactly the same spot. Is there some way we can verify whether this is the ongoing Mac test flakiness or a real problem?

[dennycd]

looks like protobuf 3.18.1 update pulled in a new gtest dependency in one of the test files (https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/reflection_tester.cc#L35), gonna test excluding this following the official spec (https://github.com/protocolbuffers/protobuf/blob/master/Protobuf-C%2B%2B.podspec#L25)

[dennycd]

protobuf build break should now be gone. Cronet ios cpp test crash appears non-related to this PR. investigating separately

[markdroth]

The remaining failures here all seem unrelated, so I'm going to go ahead and merge.