ResourceExhausted error while querying longer duration of data

[utkarshGraas]

Describe the bug
Error while querying longer duration of data. Below is the exact error message: Error (rpc error: code = ResourceExhausted desc = grpc: received message after decompression larger than max 16777216 ). Please check the server logs for more details.

To Reproduce
Steps to reproduce the behavior:

1. Trace any service that has good frequency of data
2. Set the span limit > 2 lakhs
3. Query for duration that fetches span more than 1 lakhs
4. Keep increasing the duration, unless you see the error

Expected behavior
The traces should be visible for at least 24hrs of data, currently breaking for 6hrs

Environment:

• Traces Captured using Tempo

Additional Context
I am trying to get the total notification count through my trace.

[joe-elliott]

You are simply exceeding the grpc server configured max size while requesting large amounts of data. Try increasing grpc_server_max_recv_msg_size and grpc_server_max_send_msg_size in the server block of your config.

https://grafana.com/docs/tempo/latest/configuration/#server

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity.
Please apply keepalive label to exempt this Issue.