ResourceExhausted

[edumotya]

I get ResourceExhausted error for some pdf files. It is weird, for instance, one of them sizes only 177.9 KB in disk (237.14 KB as a base64 string). The error says "ResourceExhausted: 429 Received message larger than max (5218782 vs. 4194304)".

Environment details

• OS type and version:
• Python version: 3.6.12
• pip version: 20.2.4
• google-cloud-documentai version: 0.3.0

Trace

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.RESOURCE_EXHAUSTED
	details = "Received message larger than max (5218782 vs. 4194304)"
	debug_error_string = "{"created":"@1612458152.165208175","description":"Received message larger than max (5218782 vs. 4194304)","file":"src/core/ext/filters/message_size/message_size_filter.cc","file_line":204,"grpc_status":8}"
>
  File "/usr/local/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "grpc/_channel.py", line 923, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "grpc/_channel.py", line 826, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
ResourceExhausted: 429 Received message larger than max (5218782 vs. 4194304)
    result = client.process_document(request=request, timeout=DOCUMENTAI_TIMEOUT)
  File "/usr/local/lib/python3.6/site-packages/google/cloud/documentai_v1beta3/services/document_processor_service/client.py", line 327, in process_document
    response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
  File "/usr/local/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
    return wrapped_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/google/api_core/retry.py", line 286, in retry_wrapped_func
    on_error=on_error,
  File "/usr/local/lib/python3.6/site-packages/google/api_core/retry.py", line 184, in retry_target
    return target()
  File "/usr/local/lib/python3.6/site-packages/google/api_core/timeout.py", line 102, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
    # Permission is hereby granted, free of charge, to any person obtaining a copy

Code example

client = documentai.DocumentProcessorServiceClient()
# find size
print(len(content.encode('utf-8'))) # 237140 bytes
request = documentai.types.ProcessRequest(
        name=name,
        document=documentai.types.Document(content=content, mime_type=mime_type),
    )
result = client.process_document(request=request, timeout=DOCUMENTAI_TIMEOUT)

[edumotya]

I think the issue is that the DocumentAI response is too large (not the request). It is similar to this googleapis/python-texttospeech#5.

Moreover, digging a bit deeper it seems that there is no limitation on the client side:

python-documentai/google/cloud/documentai_v1beta3/services/document_processor_service/transports/grpc.py

Line 179 in 8d8fc5a

("grpc.max_receive_message_length", -1),

Therefore, the limitation should be on the server side.

[edumotya]

I am sorry. It seems that the v0.4.0 release solves this issue.

pip install git+https://github.com/googleapis/python-documentai@release-v0.4.0