Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

target-type:pod not working #469

Closed
cjbottaro opened this issue Jul 18, 2018 · 3 comments
Closed

target-type:pod not working #469

cjbottaro opened this issue Jul 18, 2018 · 3 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@cjbottaro
Copy link

Hello again!

{{- $values := index .Values .Values.region .Values.env -}}

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: load-balancer
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/certificate-arn: {{ $values.cert_arn }}
    alb.ingress.kubernetes.io/target-type: pod
    alb.ingress.kubernetes.io/subnets: {{ $values.subnets | join "," }}
    alb.ingress.kubernetes.io/security-groups: {{ $values.security_groups | join "," }}
    alb.ingress.kubernetes.io/healthcheck-path: /ping
spec:
  rules:
    - host: "*.{{ $values.domain }}"
      http:
        paths:
          - path: /api
            backend:
              serviceName: sa-api
              servicePort: 80
          - backend:
              serviceName: sa-web
              servicePort: 80

I'm surprised that only one target group is being made:
http://storage.stochasticbytes.com.s3.amazonaws.com/8wk4YEs0.png

I would have guessed that a target group for each service would have been made.

I have pods from two different services:

$ kubectl get pods -l base=sa -o wide
NAME                      READY     STATUS    RESTARTS   AGE       IP            NODE
sa-api-5c7cb576df-4gwsz   1/1       Running   0          1d        10.3.20.124   ip-10-3-20-7.ec2.internal
sa-web-68bc5f44f4-996vt   1/1       Running   0          1d        10.3.20.65    ip-10-3-20-7.ec2.internal

But both of those IP's (10.3.20.124 and 10.3.20.65) show up in the target group's targets. And they are both either unhealthy or draining.

Here are the ingress controller logs:

$ kubectl logs -f --namespace kube-system pod/alb-ingress-controller-7778867b87-q9g7q
I0718 17:08:43.064212       1 main.go:24] [ALB-INGRESS] [controller] [INFO]: Log level read as "", defaulting to INFO. To change, set LOG_LEVEL environment variable to WARN, ERROR, or DEBUG.
I0718 17:08:43.064472       1 launch.go:112] &{ALB Ingress Controller 1.0-beta.4 git-d90522dc git://github.com/kubernetes-sigs/aws-alb-ingress-controller}
I0718 17:08:43.064489       1 launch.go:115] Watching for ingress class: alb
I0718 17:08:43.064612       1 launch.go:282] Creating API client for https://172.20.0.1:443
I0718 17:08:43.081192       1 launch.go:294] Running in Kubernetes Cluster version v1.10 (v1.10.3) - git (clean) commit 2bba0127d85d5a46ab4b778548be28623b32d0b0 - platform linux/amd64
I0718 17:08:43.086352       1 launch.go:134] validated kube-system/default-http-backend as the default backend
I0718 17:08:43.095895       1 alb-controller.go:121] [ALB-INGRESS] [controller] [INFO]: Ingress class set to alb
I0718 17:08:43.095909       1 alb-controller.go:130] [ALB-INGRESS] [controller] [INFO]: albNamePrefix undefined, defaulting to 7506d366
I0718 17:08:43.352148       1 albingresses.go:83] [ALB-INGRESS] [ingress] [INFO]: Building list of existing ALBs
I0718 17:08:43.514434       1 albingresses.go:91] [ALB-INGRESS] [ingress] [INFO]: Fetching information on 0 ALBs
I0718 17:08:43.549389       1 albingresses.go:108] [ALB-INGRESS] [ingress] [INFO]: Assembled 0 ingresses from existing AWS resources in 197.20963ms
I0718 17:08:43.549450       1 controller.go:1359] starting Ingress controller
I0718 17:08:43.551942       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/load-balancer
I0718 17:08:43.921703       1 leaderelection.go:174] attempting to acquire leader lease...
I0718 17:08:43.921875       1 controller.go:477] backend reload required
I0718 17:08:43.927201       1 status.go:193] new leader elected: alb-ingress-controller-69c85985-dg84x
I0718 17:08:44.906445       1 loadbalancer.go:306] [ALB-INGRESS] [default/load-balancer] [INFO]: Start ELBV2 creation.
I0718 17:08:45.413382       1 loadbalancer.go:313] [ALB-INGRESS] [default/load-balancer] [INFO]: Completed ELBV2 creation. Name: 7506d366-default-loadbalan-ed96 | ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:loadbalancer/app/7506d366-default-loadbalan-ed96/363843c35ac4def0
I0718 17:08:45.413417       1 targetgroup.go:185] [ALB-INGRESS] [default/load-balancer] [INFO]: Start TargetGroup creation.
I0718 17:08:45.413481       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' 7506d366-default-loadbalan-ed96 created
I0718 17:08:45.931701       1 targetgroup.go:190] [ALB-INGRESS] [default/load-balancer] [INFO]: Succeeded TargetGroup creation. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f | Name: 7506d366-05001-HTTP-76ce687.
I0718 17:08:45.931723       1 targetgroup.go:185] [ALB-INGRESS] [default/load-balancer] [INFO]: Start TargetGroup creation.
I0718 17:08:45.931799       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' 7506d366-05001-HTTP-76ce687 target group created
I0718 17:08:46.295372       1 targetgroup.go:190] [ALB-INGRESS] [default/load-balancer] [INFO]: Succeeded TargetGroup creation. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f | Name: 7506d366-05001-HTTP-76ce687.
I0718 17:08:46.295390       1 listener.go:140] [ALB-INGRESS] [default/load-balancer] [INFO]: Start Listener creation.
I0718 17:08:46.295466       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' 7506d366-05001-HTTP-76ce687 target group created
I0718 17:08:46.707749       1 listener.go:145] [ALB-INGRESS] [default/load-balancer] [INFO]: Completed Listener creation. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:listener/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb | Port: 443 | Proto: HTTPS.
I0718 17:08:46.707769       1 rule.go:121] [ALB-INGRESS] [default/load-balancer] [INFO]: Start Rule creation.
I0718 17:08:46.707843       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' 443 listener created
I0718 17:08:46.781413       1 rule.go:126] [ALB-INGRESS] [default/load-balancer] [INFO]: Completed Rule creation. Rule Priority: "1" | Condition: [{    Field: "host-header",    Values: ["*.dacademicworks.com"]  },{    Field: "path-pattern",    Values: ["/api"]  }]
I0718 17:08:46.781436       1 rule.go:121] [ALB-INGRESS] [default/load-balancer] [INFO]: Start Rule creation.
I0718 17:08:46.781473       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' 1 rule created
I0718 17:08:46.803470       1 rule.go:126] [ALB-INGRESS] [default/load-balancer] [INFO]: Completed Rule creation. Rule Priority: "2" | Condition: [{    Field: "host-header",    Values: ["*.dacademicworks.com"]  }]
I0718 17:08:46.803549       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3197387", FieldPath:""}): type: 'Normal' reason: 'CREATE' 2 rule created
I0718 17:08:46.803632       1 controller.go:486] ingress backend successfully reloaded...
I0718 17:09:17.406337       1 leaderelection.go:184] successfully acquired lease kube-system/ingress-controller-leader-alb
I0718 17:09:17.406377       1 status.go:193] new leader elected: alb-ingress-controller-7778867b87-q9g7q
I0718 17:09:43.946570       1 status.go:337] updating Ingress default/load-balancer status to [{ 7506d366-default-loadbalan-ed96-198693280.us-east-1.elb.amazonaws.com}]
I0718 17:09:43.952987       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/load-balancer
I0718 17:09:53.550209       1 alb-controller.go:154] [ALB-INGRESS] [controller] [INFO]: Ingress update being attempted. (Forced from no event seen in 60 seconds).
I0718 17:09:53.700826       1 loadbalancer.go:549] [ALB-INGRESS] [default/load-balancer] [INFO]: Modifying ELBV2 tags to [  {    Key: "IngressName",    Value: "load-balancer"  },  {    Key: "Namespace",    Value: "default"  },  {    Key: "kubernetes.io/cluster/edge",    Value: "owned"  },  {    Key: "kubernetes.io/ingress-name",    Value: "default/load-balancer"  }].
I0718 17:09:53.752175       1 targetgroup.go:293] [ALB-INGRESS] [default/load-balancer] [INFO]: Modifying target group tags.
I0718 17:09:53.752196       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Normal' reason: 'MODIFY' 7506d366-default-loadbalan-ed96 tags modified
I0718 17:09:53.791774       1 targetgroup.go:303] [ALB-INGRESS] [default/load-balancer] [INFO]: Modifying target group targets.
E0718 17:09:54.079254       1 rule.go:143] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to locate TargetGroup related to this service: sa-web:80
I0718 17:09:54.079392       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Normal' reason: 'MODIFY' 7506d366-05001-HTTP-76ce687 target group modified
I0718 17:09:54.096304       1 targetgroup.go:169] [ALB-INGRESS] [default/load-balancer] [INFO]: Start TargetGroup deletion. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f | Name: 7506d366-05001-HTTP-76ce687.
I0718 17:09:54.096329       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
	status code: 400, request id: 63a84ed1-8aad-11e8-b236-995431cef23b
E0718 17:11:34.815512       1 albingress.go:264] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to reconcile state on this ingress
E0718 17:11:34.815535       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]:  - Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
E0718 17:11:34.815541       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]: 	status code: 400, request id: 63a84ed1-8aad-11e8-b236-995431cef23b
E0718 17:11:34.815545       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]:  - Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f
I0718 17:11:34.815584       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error deleting 7506d366-05001-HTTP-76ce687 target group: Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f
I0718 17:11:44.815702       1 alb-controller.go:154] [ALB-INGRESS] [controller] [INFO]: Ingress update being attempted. (Forced from no event seen in 60 seconds).
E0718 17:11:44.961944       1 rule.go:143] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to locate TargetGroup related to this service: sa-web:80
I0718 17:11:45.030605       1 targetgroup.go:169] [ALB-INGRESS] [default/load-balancer] [INFO]: Start TargetGroup deletion. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f | Name: 7506d366-05001-HTTP-76ce687.
I0718 17:11:45.030646       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
	status code: 400, request id: a5c200ba-8aad-11e8-8504-39989e2ad32d
E0718 17:13:25.709008       1 albingress.go:264] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to reconcile state on this ingress
E0718 17:13:25.709028       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]:  - Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
E0718 17:13:25.709034       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]: 	status code: 400, request id: a5c200ba-8aad-11e8-8504-39989e2ad32d
E0718 17:13:25.709039       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]:  - Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f
I0718 17:13:25.709085       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error deleting 7506d366-05001-HTTP-76ce687 target group: Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f
I0718 17:13:35.709176       1 alb-controller.go:154] [ALB-INGRESS] [controller] [INFO]: Ingress update being attempted. (Forced from no event seen in 60 seconds).
E0718 17:13:35.925831       1 rule.go:143] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to locate TargetGroup related to this service: sa-web:80
I0718 17:13:35.960897       1 targetgroup.go:169] [ALB-INGRESS] [default/load-balancer] [INFO]: Start TargetGroup deletion. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f | Name: 7506d366-05001-HTTP-76ce687.
I0718 17:13:35.960904       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
	status code: 400, request id: e7e57577-8aad-11e8-9338-79963b75b1f8
E0718 17:15:16.619022       1 albingress.go:264] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to reconcile state on this ingress
E0718 17:15:16.619048       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]:  - Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
E0718 17:15:16.619052       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]: 	status code: 400, request id: e7e57577-8aad-11e8-9338-79963b75b1f8
E0718 17:15:16.619057       1 albingress.go:266] [ALB-INGRESS] [default/load-balancer] [ERROR]:  - Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f
I0718 17:15:16.619102       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error deleting 7506d366-05001-HTTP-76ce687 target group: Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f
I0718 17:15:26.619171       1 alb-controller.go:154] [ALB-INGRESS] [controller] [INFO]: Ingress update being attempted. (Forced from no event seen in 60 seconds).
E0718 17:15:26.775932       1 rule.go:143] [ALB-INGRESS] [default/load-balancer] [ERROR]: Failed to locate TargetGroup related to this service: sa-web:80
I0718 17:15:26.809322       1 targetgroup.go:169] [ALB-INGRESS] [default/load-balancer] [INFO]: Start TargetGroup deletion. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/aeea8a371d4d4f9f | Name: 7506d366-05001-HTTP-76ce687.
I0718 17:15:26.809343       1 event.go:218] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions", ResourceVersion:"3439863", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/363843c35ac4def0/17ce5a2823ee4aeb/c39c0eccd95b6262: ValidationError: A target group ARN must be specified
	status code: 400, request id: 29f64d35-8aae-11e8-92f7-85ccd5ac1eff

Thanks again!
@bigkraig
Copy link

This was fixed after the 1.0-beta.4 tag was cut, i'll have another tag after #476 is merged or you can try out that branch if this is an immediate issue.

@cjbottaro
Copy link
Author

Still not working using latest (3e303f7)...

$ kubectl logs --namespace kube-system pod/alb-ingress-controller-5589585dfb-dsszc -f
-------------------------------------------------------------------------------
AWS ALB Ingress controller
  Release:    1.0-beta.4
  Build:      git-3e303f71
  Repository: https://github.com/kubernetes-sigs/aws-alb-ingress-controller
-------------------------------------------------------------------------------

I0726 20:10:46.060065       1 flags.go:131] Watching for Ingress class: alb
W0726 20:10:46.060171       1 flags.go:149] Environment variable configuration is deprecated, switch to the --cluster-name flag.
W0726 20:10:46.060180       1 flags.go:177] Environment variable configuration is deprecated, switch to the --aws-max-retries flag.
W0726 20:10:46.060190       1 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0726 20:10:46.060311       1 main.go:159] Creating API client for https://172.20.0.1:443
I0726 20:10:46.078887       1 main.go:203] Running in Kubernetes cluster version v1.10 (v1.10.3) - git (clean) commit 2bba0127d85d5a46ab4b778548be28623b32d0b0 - platform linux/amd64
I0726 20:10:46.079525       1 alb.go:80] ALB resource names will be prefixed with 7506d366
I0726 20:10:46.090500       1 alb.go:163] Starting AWS ALB Ingress controller
I0726 20:10:46.318361       1 albingresses.go:74] Building list of existing ALBs
I0726 20:10:46.517082       1 albingresses.go:82] Fetching information on 0 ALBs
I0726 20:10:46.570533       1 albingresses.go:100] Assembled 0 ingresses from existing AWS resources in 252.150624ms
I0726 20:10:47.770983       1 leaderelection.go:185] attempting to acquire leader lease  kube-system/ingress-controller-leader-alb...
I0726 20:10:47.777190       1 status.go:152] new leader elected: alb-ingress-controller-5589585dfb-dx9zq
I0726 20:10:47.964096       1 loadbalancer.go:306] default/load-balancer: Start ELBV2 creation.
I0726 20:10:48.659450       1 loadbalancer.go:313] default/load-balancer: Completed ELBV2 creation. Name: 7506d366-default-loadbalan-ed96 | ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:loadbalancer/app/7506d366-default-loadbalan-ed96/57f461ddbb174be5
I0726 20:10:48.659468       1 targetgroup.go:189] default/load-balancer: Start TargetGroup creation.
I0726 20:10:48.659568       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'CREATE' 7506d366-default-loadbalan-ed96 created
I0726 20:10:49.039487       1 targetgroup.go:194] default/load-balancer: Succeeded TargetGroup creation. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/6822dca1713d02ab | Name: 7506d366-05001-HTTP-76ce687.
I0726 20:10:49.039508       1 targetgroup.go:189] default/load-balancer: Start TargetGroup creation.
I0726 20:10:49.039582       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'CREATE' 7506d366-05001-HTTP-76ce687 target group created
I0726 20:10:49.420380       1 targetgroup.go:194] default/load-balancer: Succeeded TargetGroup creation. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/6822dca1713d02ab | Name: 7506d366-05001-HTTP-76ce687.
I0726 20:10:49.420399       1 listener.go:140] default/load-balancer: Start Listener creation.
I0726 20:10:49.420469       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'CREATE' 7506d366-05001-HTTP-76ce687 target group created
I0726 20:10:49.906090       1 listener.go:145] default/load-balancer: Completed Listener creation. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:listener/app/7506d366-default-loadbalan-ed96/57f461ddbb174be5/f3051ec730990122 | Port: 443 | Proto: HTTPS.
I0726 20:10:49.906110       1 rule.go:121] default/load-balancer: Start Rule creation.
I0726 20:10:49.906152       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'CREATE' 443 listener created
I0726 20:10:49.928164       1 rule.go:126] default/load-balancer: Completed Rule creation. Rule Priority: "1" | Condition: [{    Field: "host-header",    Values: ["*.dacademicworks.com"]  },{    Field: "path-pattern",    Values: ["/api"]  }]
I0726 20:10:49.928182       1 rule.go:121] default/load-balancer: Start Rule creation.
I0726 20:10:49.928242       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'CREATE' 1 rule created
I0726 20:10:50.029832       1 rule.go:126] default/load-balancer: Completed Rule creation. Rule Priority: "2" | Condition: [{    Field: "host-header",    Values: ["*.dacademicworks.com"]  },{    Field: "path-pattern",    Values: ["/"]  }]
I0726 20:10:50.029926       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'CREATE' 2 rule created
I0726 20:10:51.104760       1 loadbalancer.go:549] default/load-balancer: Modifying ELBV2 tags to [  {    Key: "IngressName",    Value: "load-balancer"  },  {    Key: "Namespace",    Value: "default"  },  {    Key: "kubernetes.io/cluster/edge",    Value: "owned"  },  {    Key: "kubernetes.io/ingress-name",    Value: "default/load-balancer"  }].
I0726 20:10:51.141353       1 targetgroup.go:297] default/load-balancer: Modifying target group tags.
I0726 20:10:51.141452       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'MODIFY' 7506d366-default-loadbalan-ed96 tags modified
I0726 20:10:51.176003       1 targetgroup.go:310] default/load-balancer: Modifying target group targets. Adding (10.3.20.65:5001) and removing (10.3.20.124:5001)
E0726 20:10:51.454198       1 rule.go:143] default/load-balancer: Failed to locate TargetGroup related to this service: sa-web:80
I0726 20:10:51.454267       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Normal' reason: 'MODIFY' 7506d366-05001-HTTP-76ce687 target group modified
I0726 20:10:51.462806       1 targetgroup.go:173] default/load-balancer: Start TargetGroup deletion. ARN: arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/6822dca1713d02ab | Name: 7506d366-05001-HTTP-76ce687.
I0726 20:10:51.462821       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/57f461ddbb174be5/f3051ec730990122/580233757a38d93d: ValidationError: A target group ARN must be specified
	status code: 400, request id: fe764201-910f-11e8-b2e4-cdaab82ef6f8
I0726 20:11:23.241152       1 leaderelection.go:194] successfully acquired lease kube-system/ingress-controller-leader-alb
I0726 20:11:23.241186       1 status.go:152] new leader elected: alb-ingress-controller-5589585dfb-dsszc
E0726 20:12:32.093845       1 albingress.go:315] default/load-balancer: Failed to reconcile state on this ingress
E0726 20:12:32.093867       1 albingress.go:317] default/load-balancer:  - Error modifying rule arn:aws:elasticloadbalancing:us-east-1:853401350379:listener-rule/app/7506d366-default-loadbalan-ed96/57f461ddbb174be5/f3051ec730990122/580233757a38d93d: ValidationError: A target group ARN must be specified
E0726 20:12:32.093874       1 albingress.go:317] default/load-balancer: 	status code: 400, request id: fe764201-910f-11e8-b2e4-cdaab82ef6f8
E0726 20:12:32.093880       1 albingress.go:317] default/load-balancer:  - Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/6822dca1713d02ab
E0726 20:12:32.093893       1 albingress.go:320] default/load-balancer: Will retry to reconcile in 1m0.299828828s
I0726 20:12:32.093942       1 event.go:221] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"load-balancer", UID:"c915abd8-8958-11e8-a69e-1254143763fe", APIVersion:"extensions/v1beta1", ResourceVersion:"4598529", FieldPath:""}): type: 'Warning' reason: 'ERROR' Error deleting 7506d366-05001-HTTP-76ce687 target group: Timed out trying to delete target group arn:aws:elasticloadbalancing:us-east-1:853401350379:targetgroup/7506d366-05001-HTTP-76ce687/6822dca1713d02ab```

@bigkraig bigkraig mentioned this issue Jul 27, 2018
Merged
@bigkraig bigkraig added the kind/bug Categorizes issue or PR as related to a bug. label Jul 27, 2018
@cjbottaro
Copy link
Author

Things are working well now 1.0-beta.5 (2894ebf). Thank you very much for your work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cjbottaro @bigkraig