Agent keeps failing and host state stucks at "Reconnecting"

[fernandoneto]

Rancher Version :

v0.59.1

Docker Version:

1.10.2

OS:

Ubuntu 14.04.4 LTS (3.13.0-79-generic)

Steps to Reproduce:

• Create a instance in AWS and launch rancher-server.
• Wait until UI is available, configure auth and then lets add a host.
• In add host menu select "Other" and fill the fields according your configurations. We are using Azure driver.
• Add some container's or a stack from catalog.

Results:

After some time the host go to Reconnecting and we have to restart rancher-server container to host go to Active again

Expected:

Hosts recover from reconnecting state automatically.

Related:

#2196

[deniseschannon]

Unable to reproduce. I launched an Azure using the Azure driver (new in v0.63.0, it has it's own dedicated page) and added a service onto it. It launched the containers fine and did not go into reconnecting. I will keep it up for the next 24 hours and try to monitor. Once it goes into reconnecting, does it recover on its own?

[fernandoneto]

@deniseschannon no, i have to restart the rancher-server.

[deniseschannon]

I am unable to reproduce. I left my box up for 24 hours and had no issues where the host went into reconnecting.

Why do you need to restart rancher-server? Couldn't you just re-add the agent?

[fernandoneto]

@deniseschannon because our production environment has many hosts. Is easier to have a bash script that restart rancher server every 2 hours and prevent reconneting.

[fernandoneto]

@deniseschannon at this moment im getting a reconneting host in my test environment. If you want i can provide you acess to this environment. just send me a email to fernando.neto@junglecloud.com

[rtorino]

I'm also experiencing the same issue. I think it started after we rebooted the rancher server host.

[deniseschannon]

@fernandoneto @rtorino

Are you still having these issues with Rancher v1.1.1?

[bacheson]

Same exact thing here. It appears to afflict setups with many containers. The Hosts UI is bordering on unusable at this point with our setup. v1.1.2

[deniseschannon]

@bacheson Would you mind opening a new issue for yours? I know the others were using Azure and it looks like you are using Packet.

I'll close this one cause of the old version of Rancher as well.