Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Network Bindings - not configured" although there are port mappings in the container. #1075

Closed
jmetzmeier opened this issue Nov 14, 2017 · 12 comments

Comments

@jmetzmeier
Copy link

This is a duplicate of #1057, #1063, and possibly #1064. #1063 was closed as fixed in version 1.15.1, however, I had a chance to test today and the problem is still present in that version.

@nmeyerhans
Copy link
Contributor

nmeyerhans commented Nov 14, 2017

Can you provide me with ecs-agent agent logs and a task ID that saw impact? Are you using the ECS Optimized AMI? The logs would be in /var/log/ecs/. Feel free to send them to me via email at nmeyerha@amazon.com rather than posting them publicly here.

@jmetzmeier
Copy link
Author

I sent the logs and task information to you via email. I am building an AMI using the ECS Optimized AMI as a base. Our only changes to the AMI is some basic user creation and the addition of a monitoring daemon (built via packer). I tested 1.5.0 with the vanilla AMI and still encountered the issue, but did not test vanilla with 1.5.1. The version we were using before I upgraded the AMI was 1.14.3, which continues to work properly.

@nmeyerhans
Copy link
Contributor

@jmetzmeier Thanks for reporting this. We were able to confirm the issue and reproduce it here, and you can see @richardpen has just proposed a fix. I don't have an ETA for including this in a release at this time, but stay tuned for updates...

@bndw
Copy link

bndw commented Nov 17, 2017

I believe I'm running into this as well with agent version 1.15.1.

  • I have portMappings defined for my container
  • They're being mapped by Docker
    0.0.0.0:32771->80/tcp
    
  • describe-tasks does not show them (note no networkBindings attr):
    {
        "containerArn": "arn:aws:ecs:us-east-2:xxx:container/b63f46d8-b45c-461b-aa56-958737b2a5c9",
        "taskArn": "arn:aws:ecs:us-east-2:xxx:task/982d52c6-8401-4fbc-b18e-76eaa09cf353",
        "lastStatus": "RUNNING",
        "name": "streamline-sidecar"
    }

This appears to be an intermittent issue, as sometimes the networkBindings show up after killing the task a few times.

EDIT: The severity of this bug is extremely high IMHO. We depend on this information for service discovery and without it our services are unreachable.

@nmeyerhans
Copy link
Contributor

@bndw Yes, it certainly sounds like it. We're releasing 1.15.2 today with a complete fix. Apologies for the inconvenience.

@bndw
Copy link

bndw commented Nov 17, 2017

Thanks for the fast response @nmeyerhans

@nmeyerhans
Copy link
Contributor

Agent 1.15.2 is available now. You can get it from Docker Hub. It's also on the latest ECS optimized AMI, 2017.09.c. The list of regional AMI IDs is available in our docs, or you can use the following command to determine the AMI ID for a given region:

aws ec2 describe-images --owner 591542846629 --region us-west-2 --filter=Name=name,Values=amzn-ami-2017.09.c-amazon-ecs-optimized

@ctaggart
Copy link

ctaggart commented Jan 8, 2018

Any idea if this bug still affects ECS Fargate?
https://forums.aws.amazon.com/thread.jspa?threadID=270940&tstart=0

@jhaynes
Copy link
Contributor

jhaynes commented Jan 9, 2018

@ctaggart You shouldn't be running into this bug on Fargate. If you are, please feel free to contact support. We can potentially try to debug here, but support will probably be a better option.

@guizmaii
Copy link

guizmaii commented Aug 7, 2018

I confirm that this bug is always present in Fargate.

@nathando
Copy link

This cannot be closed because it still happens. Fargate does not report network bindings correctly at all. We need this information

@nietzscheson
Copy link

This cannot be closed because it still happens. Fargate does not report network bindings correctly at all. We need this information

Yes, same for here!

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants