AWS SNS event source crash with error "http: panic serving : runtime error: invalid memory address or nil pointer dereference"

[sanandan]

Describe the bug

To Reproduce
Steps to reproduce the behavior:

1. deploy aws sns source, example provided.
2. setup egress(aws load balancer) on aws sns service
3. As soon as aws health check hits the 13000 port the service crashes with panic error.
4. Even without setting up ingress if we curl the sns-event-source service on the port crashes the application.

Expected behavior
HTTP Health check or curl to sns-event-source service should return 200. So the pod can register as healthy under the target group to confirm subscription from SNS

Screenshots

Environment (please complete the following information):

• Kubernetes: v1.19
• Argo Events: v1.3.0

Additional context
Add any other context about the problem here.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[sanandan]

Closing it as it was configuration issue on ingress. I had not mentioned health check path /health. After updating to reflect as mentioned in the doc its registering healthy as expected

[whynowy]

@sanandan - thanks for the update, glad to hear it works for you! Even though you are good with it, I want to confirm that when you hit the endpoint with a curl, the pod panics, right? I think that's also an issue we need to fix.

[sanandan]

yes when i curl -eventsource-svc:12000 the panic happened.

[sanandan]

@whynowy Please let me know if you need me to reopen this issue.
Also how can enable debug log on eventsource container. It was able tosuccessfully register for subscription, but failed to confirm the subscription.

[whynowy]

@sanandan - thanks, I already put a fix there

[whynowy]

@whynowy Please let me know if you need me to reopen this issue.
Also how can enable debug log on eventsource container. It was able tosuccessfully register for subscription, but failed to confirm the subscription.

Put environment variable DEBUG_LOG: "true" in spec.template.container.envs

[sanandan]

@whynowy I'm able to set debug_log env variable for eventsource-controller deployments and logs are showing all levels. But since the sns event source deployment is created by eventsource-controller, i'm not able to setup the logs for debug level.

[whynowy]

Setting the env in EventSource spec.template.container.envs, the pod will be created with that env.

[sanandan]

apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
  name: aws-sns
spec:
  service:
    ports:
      - port: 12000
        targetPort: 12000
  sns:
    snowemail:
      topicArn: arn:aws:sns:us-east-1:xxxxx:sns-argo-events
      webhook:
        endpoint: "/"
        port: "12000"
        method: POST
        url: https://xxxxx.execute-api.us-east-1.amazonaws.com
      region: us-east-1
  template:
    container:
      env:
       - name: DEBUG_LOG
         value: 'true'

Even still not getting full log. If i can the subscribe url from the sns confirmation msg then , i should be maually confirm the subscription and test the sns events. Unable to proceed further. The SNS subscription shows as pending.

[whynowy]

Do you have the URL https://xxxxx.execute-api.us-east-1.amazonaws.com created in advance?

The logic here is kind of tricky. service used in the spec is only for testing purpose for webhook typed event sources (such as webhook, github, sns, and so on), see the detail at https://github.com/argoproj/argo-events/blob/master/docs/eventsources/services.md.

So for sns, a LB service (or an Ingress) is needed, the service or ingress needs to point to the event source pod. I think pending status you see is because the url is not accessible.

[whynowy]

And you can reference to your current ClusterIP type service to create a LB Service (labelSelectors), and remove service field from the EventSource spec.

[sanandan]

Thankyou @whynowy it was sec group config issue. i'm able to publish and view the messages in the log :)

[whynowy]

Thankyou @whynowy it was sec group config issue. i'm able to publish and view the messages in the log :)

👍