Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config grpc_server_max_recv_msg_size not work #2437

Closed
lovesharepc opened this issue May 7, 2023 · 2 comments
Closed

config grpc_server_max_recv_msg_size not work #2437

lovesharepc opened this issue May 7, 2023 · 2 comments

Comments

@lovesharepc
Copy link

Describe the bug
add config grpc_server_max_recv_msg_size notwork
config

  grpc_server_max_recv_msg_size: 104857600
  grpc_server_max_send_msg_size: 104857600

opentelmetry collector error log

error   exporterhelper/queued_retry.go:401      Exporting failed. The error is not retryable. Dropping data.    {"kind": "exporter", "data_type": "traces", "name": "otlp", "error": "Permanent error: rpc error: code = ResourceExhausted desc = grpc: received message after decompression larger than max (5089930 vs. 4194304)", "dropped_items": 10003}

To Reproduce
Steps to reproduce the behavior:

  1. Start Tempo (SHA or version)
owan@NB:/data/grafana_stack$ dc exec tempo-compactor-1 ./tempo --version
tempo, version 2.1.1 (branch: HEAD, revision: 4157d7620)
  build user:       
  build date:       
  go version:       go1.20.3
  platform:         linux/amd64
  tags:             unknown
  1. Perform Operations (Write)

Expected behavior

Environment:

  • Infrastructure: [docker]
  • Deployment tool: [docker-compose]

Additional Context

@joe-elliott
Copy link
Member

The GRPC server that serves the OTLP endpoint is unfortunately configured differently then the GRPC servers that Tempo uses to talk to itself. The configuration option you are referencing controls our internal GRPC server. To configure the OTLP endpoint use the distributors block.

I believe this will work:

distributor:
  receivers:
    otlp:
      grpc:
         max_recv_msg_size_mib: ??

here are the valid configuration options:
https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configgrpc/README.md

@lovesharepc
Copy link
Author

oh, I got, thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants