-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GRPC Receiver Timeout in Python Lambda running in a Container #209
Comments
Is it possible that a role needs to be assigned to the lambda to allow the GRPC receiver to start up/connect? I haven't seen any thing confirming that but want to rule it out. NOTE: I'm not trying to call any AWS services yet, just get the layer working inside a container to do basic logging. |
Circling back on this, I've discovered that if the AWS_LAMBDA_EXEC_WRAPPER environment variable is set, the exhibited behavior is the timeout starting up the receiver. I deleted this env variable on a whim, leaving the OPENTELEMETRY_COLLECTOR_CONFIG_FILE env var intact and I now have logs being written. However, I still do not have any events going to external exporters (Honeycomb specifically for some reason. I'm going to try and set up xray and see if the behavior is the same. |
I've confirmed that the extension is running now in the container but logs are never sent to external exporters. Logging and x-ray work fine but my Honeycomb exporter does not send events. I'm sort of at a loss for how to debug this and welcome any insight.
Dockerfile
Lambda is built and deployed via SAM. |
Turns out, I had a mistake in the value for AWS_LAMBDA_EXEC_WRAPPER. This was pointing to a non-existent location/script. Once I pointed this at /opt/otel-instrument, traces started flowing. Sorry for the trouble. You can close this issue. |
Thanks for confirming @osiris43 |
I have a Python lambda deployed and running in a container, similar to this issue "publish Docker images". I have managed to compile and place the aws-otel-lambda extension in my container correctly as far as I can tell. However, when I execute the function, the lambda times out waiting for the gRPC receiver to start. The timeout is set to 30 seconds.
My collector config:
I have a hunch I need to do something else with my container to enable this but not sure what. Any thoughts would be appreciated.
The text was updated successfully, but these errors were encountered: