Generated `test_*_flattened_error` test fails when the first field of a request message is a nested message

[parthea]

Generated unit tests, specifically test_*_flattened_error, for the following proto file fail with error below

syntax = "proto3";

package google.fragment;

import "google/protobuf/struct.proto";
import "google/api/client.proto";

service MyService {
  option (google.api.default_host) = "my.example.com";

  rpc MyMethod(MethodRequest) returns (MethodResponse) {
    option (google.api.method_signature) = "some_message";
  }
}

message AnotherMessage {};

message SomeMessage{
  AnotherMessage another_message = 1;
}

message MethodRequest {
  SomeMessage some_message = 1;
}

message MethodResponse {
  google.protobuf.Value result = 1;
}

(py392) partheniou@partheniou-vm-3:~/git/gapic-generator-python$ nox -s fragment-3.9
nox > Running session fragment-3.9
nox > Creating virtual environment (virtualenv) using python3.9 in .nox/fragment-3-9
nox > python -m pip install coverage pytest pytest-cov pytest-xdist 'asyncmock; python_version < '"'"'3.8'"'"'' pytest-asyncio grpcio-tools
nox > python -m pip install -e .
nox > python -m grpc_tools.protoc --proto_path=tests/fragments --python_gapic_out=/tmp/tmp7ygc3kd_ --python_gapic_opt=transport=grpc+rest,python-gapic-templates=DEFAULT test_nested_messages.proto
nox > python -m pip install /tmp/tmp7ygc3kd_ -e . -qqq -r /tmp/tmp7ygc3kd_/testing/constraints-3.9.txt
nox > py.test --quiet --cov-config=/tmp/tmp7ygc3kd_/.coveragerc --cov-report=term --cov-fail-under=100 /tmp/tmp7ygc3kd_/tests/unit
nox > Command py.test --quiet --cov-config=/tmp/tmp7ygc3kd_/.coveragerc --cov-report=term --cov-fail-under=100 /tmp/tmp7ygc3kd_/tests/unit failed with exit code 1:
/usr/local/google/home/partheniou/git/gapic-generator-python/.nox/fragment-3-9/lib/python3.9/site-packages/pytest_asyncio/plugin.py:208: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset.
The event loop scope for asynchronous fixtures will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session"

  warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET))
.............................................F.F........................ [ 64%]
.......................................                                  [100%]
=================================== FAILURES ===================================
________________________ test_my_method_flattened_error ________________________

args = (,)
kwargs = {'metadata': [('x-goog-api-client', 'gl-python/3.9.16 grpc/1.68.0 gax/2.23.0 gapic/0.0.0')]}

    @functools.wraps(callable_)
    def error_remapped_callable(*args, **kwargs):
        try:
>           return callable_(*args, **kwargs)

.nox/fragment-3-9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py:76: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/fragment-3-9/lib/python3.9/site-packages/grpc/_channel.py:1181: in __call__
    return _end_unary_response_blocking(state, call, False, None)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

state = <grpc._channel._RPCState object at 0x7fe99c605a30>
call = <grpc._cython.cygrpc.SegregatedCall object at 0x7fe99c5c8400>
with_call = False, deadline = None

    def _end_unary_response_blocking(
        state: _RPCState,
        call: cygrpc.SegregatedCall,
        with_call: bool,
        deadline: Optional[float],
    ) -> Union[ResponseType, Tuple[ResponseType, grpc.Call]]:
        if state.code is grpc.StatusCode.OK:
            if with_call:
                rendezvous = _MultiThreadedRendezvous(state, call, None, deadline)
                return state.response, rendezvous
            else:
                return state.response
        else:
>           raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
E           grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
E               status = StatusCode.UNAVAILABLE
E               details = "DNS resolution failed for my.example.com:443: C-ares status is not ARES_SUCCESS qtype=A name=my.example.com is_balancer=0: Domain name not found"
E               debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-11-22T17:06:57.241431704+00:00", grpc_status:14, grpc_message:"DNS resolution failed for my.example.com:443: C-ares status is not ARES_SUCCESS qtype=A name=my.example.com is_balancer=0: Domain name not found"}"
E           >

.nox/fragment-3-9/lib/python3.9/site-packages/grpc/_channel.py:1006: _InactiveRpcError

The above exception was the direct cause of the following exception:

    def test_my_method_flattened_error():
        client = MyServiceClient(
            credentials=ga_credentials.AnonymousCredentials(),
        )
    
        # Attempting to call a method with both a request object and flattened
        # fields is an error.
        with pytest.raises(ValueError):
>           client.my_method(
                test_nested_messages.MethodRequest(),
                some_message=test_nested_messages.SomeMessage(another_message=None),
            )

/tmp/tmp7ygc3kd_/tests/unit/gapic/fragment/test_my_service.py:911: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/tmp7ygc3kd_/google/fragment/services/my_service/client.py:614: in my_method
    response = rpc(
.nox/fragment-3-9/lib/python3.9/site-packages/google/api_core/gapic_v1/method.py:131: in __call__
    return wrapped_func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (,)
kwargs = {'metadata': [('x-goog-api-client', 'gl-python/3.9.16 grpc/1.68.0 gax/2.23.0 gapic/0.0.0')]}

    @functools.wraps(callable_)
    def error_remapped_callable(*args, **kwargs):
        try:
            return callable_(*args, **kwargs)
        except grpc.RpcError as exc:
>           raise exceptions.from_grpc_error(exc) from exc
E           google.api_core.exceptions.ServiceUnavailable: 503 DNS resolution failed for my.example.com:443: C-ares status is not ARES_SUCCESS qtype=A name=my.example.com is_balancer=0: Domain name not found

.nox/fragment-3-9/lib/python3.9/site-packages/google/api_core/grpc_helpers.py:78: ServiceUnavailable
_____________________ test_my_method_flattened_error_async _____________________

self = <google.api_core.grpc_helpers_async._WrappedUnaryUnaryCall object at 0x7fe99bdf2340>

    def __await__(self) -> Iterator[P]:
        try:
>           response = yield from self._call.__await__()

.nox/fragment-3-9/lib/python3.9/site-packages/google/api_core/grpc_helpers_async.py:85: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <_AioCall of RPC that terminated with:
        status = DNS resolution failed for my.example.com:443: C-ares status is not AR...ample.com is_balancer=0: Domain name not found", grpc_status:14, created_time:"2024-11-22T17:06:57.480035143+00:00"}"
>

    def __await__(self) -> Generator[Any, None, ResponseType]:
        """Wait till the ongoing RPC request finishes."""
        try:
            response = yield from self._call_response
        except asyncio.CancelledError:
            # Even if we caught all other CancelledError, there is still
            # this corner case. If the application cancels immediately after
            # the Call object is created, we will observe this
            # `CancelledError`.
            if not self.cancelled():
                self.cancel()
            raise
    
        # NOTE(lidiz) If we raise RpcError in the task, and users doesn't
        # 'await' on it. AsyncIO will log 'Task exception was never retrieved'.
        # Instead, if we move the exception raising here, the spam stops.
        # Unfortunately, there can only be one 'yield from' in '__await__'. So,
        # we need to access the private instance variable.
        if response is cygrpc.EOF:
            if self._cython_call.is_locally_cancelled():
                raise asyncio.CancelledError()
            else:
>               raise _create_rpc_error(
                    self._cython_call._initial_metadata,
                    self._cython_call._status,
                )
E               grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
E                       status = StatusCode.UNAVAILABLE
E                       details = "DNS resolution failed for my.example.com:443: C-ares status is not ARES_SUCCESS qtype=A name=my.example.com is_balancer=0: Domain name not found"
E                       debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"DNS resolution failed for my.example.com:443: C-ares status is not ARES_SUCCESS qtype=A name=my.example.com is_balancer=0: Domain name not found", grpc_status:14, created_time:"2024-11-22T17:06:57.480035143+00:00"}"
E               >

.nox/fragment-3-9/lib/python3.9/site-packages/grpc/aio/_call.py:327: AioRpcError

The above exception was the direct cause of the following exception:

    @pytest.mark.asyncio
    async def test_my_method_flattened_error_async():
        client = MyServiceAsyncClient(
            credentials=async_anonymous_credentials(),
        )
    
        # Attempting to call a method with both a request object and flattened
        # fields is an error.
        with pytest.raises(ValueError):
>           await client.my_method(
                test_nested_messages.MethodRequest(),
                some_message=test_nested_messages.SomeMessage(another_message=None),
            )

/tmp/tmp7ygc3kd_/tests/unit/gapic/fragment/test_my_service.py:953: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/tmp/tmp7ygc3kd_/google/fragment/services/my_service/async_client.py:307: in my_method
    response = await rpc(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <google.api_core.grpc_helpers_async._WrappedUnaryUnaryCall object at 0x7fe99bdf2340>

    def __await__(self) -> Iterator[P]:
        try:
            response = yield from self._call.__await__()
            return response
        except grpc.RpcError as rpc_error:
>           raise exceptions.from_grpc_error(rpc_error) from rpc_error
E           google.api_core.exceptions.ServiceUnavailable: 503 DNS resolution failed for my.example.com:443: C-ares status is not ARES_SUCCESS qtype=A name=my.example.com is_balancer=0: Domain name not found

.nox/fragment-3-9/lib/python3.9/site-packages/google/api_core/grpc_helpers_async.py:88: ServiceUnavailable
=============================== warnings summary ===============================
tests/unit/gapic/fragment/test_my_service.py: 16 warnings
  /tmp/tmp7ygc3kd_/google/fragment/services/my_service/client.py:258: DeprecationWarning: get_mtls_endpoint_and_cert_source is deprecated. Use the api_endpoint property instead.
    warnings.warn("get_mtls_endpoint_and_cert_source is deprecated. Use the api_endpoint property instead.",

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED ../../../../../../../tmp/tmp7ygc3kd_/tests/unit/gapic/fragment/test_my_service.py::test_my_method_flattened_error
FAILED ../../../../../../../tmp/tmp7ygc3kd_/tests/unit/gapic/fragment/test_my_service.py::test_my_method_flattened_error_async
2 failed, 109 passed, 16 warnings in 1.59s

The test expects ValueError when both request and individual fields are set on the client method.

        with pytest.raises(ValueError):
>           await client.my_method(
                test_nested_messages.MethodRequest(),
                some_message=test_nested_messages.SomeMessage(another_message=None),
            )

An exception should be raised if both request and a field are set as per the following code.

gapic-generator-python/gapic/templates/%namespace/%name_%version/%sub/services/%service/_client_macros.j2

Lines 91 to 96 in 5a37094

	# - Quick check: If we got a request object, we should *not* have
	# gotten any keyword arguments that map to the request.
	has_flattened_params = any([{{ method.flattened_fields.values()|join(", ", attribute="name") }}])
	if request is not None and has_flattened_params:
	raise ValueError('If the `request` argument is set, then none of '
	'the individual field arguments should be set.')

any([{{ method.flattened_fields.values()|join(", ", attribute="name") }}]) in the code above is checking for non-zero, but we should be checking if both request and a flattened field are not None.

[vchudnov-g]

We should get back to this if it's still occurring.

[ilai-deutel]

We should get back to this if it's still occurring.

Hi Victor, this issue is still occurring for us