Overnight test leaves some containers as Created

[lcastellano]

VIC version:
1.4.3/ master / 19788ca

Deployment details:

Three nodes vCenter with vSAN, local system in SFO

Steps to reproduce:

Run long term test to create containers with the attached script, with argument: 30

scripts.zip

Actual behavior:

Output of "docker ps -a" after killing the test and waiting for 10 mins:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
be117574d30b        ubuntu              "/bin/sh -c echo s..."   18 hours ago        Created                                 contvm_9
ee9dc9793aae        ubuntu              "/bin/sh -c echo s..."   19 hours ago        Created                                 contvm_17
23219f31869f        ubuntu              "/bin/sh -c echo s..."   19 hours ago        Created                                 contvm_7
26f782102a8a        ubuntu              "/bin/sh -c echo s..."   20 hours ago        Created                                 contvm_18
7ceea2db6cf2        ubuntu              "/bin/sh -c echo s..."   21 hours ago        Created                                 contvm_8
6e001f427e5b        ubuntu              "/bin/sh -c echo s..."   21 hours ago        Created                                 contvm_5

Expected behavior:
All containers should have been started and exited correctly

Logs:

container-logs.zip

[lcastellano]

I have done some more investigation on the set of container that did not start. They all have created correctly. The normal sequence for a "docker run" request appears to be the following:

- ContainerCreate [BEGIN]
- ContainerCreate [END]
- ContainerAttach [BEGIN]
- ContainerStart [BEGIN]
- ContainerStart [END]
- ContainerAttach [END]

In 4 of the above cases the containers have started an ATTACH and the attach has reported errors: context deadline exceeded . There was not a call to START. The other two containers had not called ATTACH after CREATE.

[renmaosheng]

will revisit once we hit this again.