New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

连接过多 #140

Closed
wu560130911 opened this Issue Dec 25, 2015 · 7 comments

Comments

Projects
None yet
4 participants
@wu560130911

wu560130911 commented Dec 25, 2015

编译安装eclipse-che-4.0.0-M6-SNAPSHOT,启动后,socket连接迅速涨到10万个,第一次请求后,tomcat报错
2015-12-25 16:32:12,226[main] [ERROR] [o.a.c.core.StandardServer 464] - StandardServer.await: accept:
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method) ~[na:1.8.0_65]
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409) ~[na:1.8.0_65]
at java.net.ServerSocket.implAccept(ServerSocket.java:545) ~[na:1.8.0_65]
at java.net.ServerSocket.accept(ServerSocket.java:513) ~[na:1.8.0_65]
at org.apache.catalina.core.StandardServer.await(StandardServer.java:446) ~[catalina.jar:8.0.28]
at org.apache.catalina.startup.Catalina.await(Catalina.java:713) [catalina.jar:8.0.28]
at org.apache.catalina.startup.Catalina.start(Catalina.java:659) [catalina.jar:8.0.28]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_65]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_65]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_65]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:351) [bootstrap.jar:8.0.28]
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:485) [bootstrap.jar:8.0.28]

@wu560130911

This comment has been minimized.

Show comment
Hide comment
@wu560130911

wu560130911 Dec 25, 2015

lsof -p 2765 | wc -l
1000073

都是socket连接

wu560130911 commented Dec 25, 2015

lsof -p 2765 | wc -l
1000073

都是socket连接

@eivantsov

This comment has been minimized.

Show comment
Hide comment
@eivantsov

eivantsov Dec 25, 2015

Looks like Docker isn;t running on your system. Can you verify that docker info returns some meaningful output?

eivantsov commented Dec 25, 2015

Looks like Docker isn;t running on your system. Can you verify that docker info returns some meaningful output?

@wu560130911

This comment has been minimized.

Show comment
Hide comment
@wu560130911

wu560130911 Dec 26, 2015

docker is already run in my system,you can see log as follows

2015-12-25 11:45:22,534[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 916] - Deploying web application archive /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war
2015-12-25 11:45:26,851[ost-startStop-1] [INFO ] [.c.p.d.c.DockerRegistryChecker 43] - Probing registry 'http://localhost:5000'
2015-12-25 11:45:26,871[ost-startStop-1] [INFO ] [.c.p.d.c.DockerRegistryChecker 48] - Probe of registry 'http://localhost:5000' succeed with HTTP response code '200'
2015-12-25 11:45:28,390[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 980] - Deployment of web application archive /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war has finished in 5,855 ms
2015-12-25 11:45:28,391[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 1036] - Deploying web application directory /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT
2015-12-25 11:45:28,429[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 1152] - Deployment of web application directory /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT has finished in 38 ms
2015-12-25 11:45:28,433[main] [INFO ] [o.a.c.http11.Http11NioProtocol 469] - Starting ProtocolHandler ["http-nio-8080"]
2015-12-25 11:45:28,461[main] [INFO ] [o.a.catalina.startup.Catalina 638] - Server startup in 70781 ms
2015-12-25 11:46:17,857[8080-Acceptor-0] [ERROR] [o.a.t.util.net.NioEndpoint 709] - Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_65]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) ~[na:1.8.0_65]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) ~[na:1.8.0_65]
at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682) ~[tomcat-coyote.jar:8.0.28]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
2015-12-25 11:46:17,908[8080-Acceptor-0] [ERROR] [o.a.t.util.net.NioEndpoint 709] - Socket accept failed

wu560130911 commented Dec 26, 2015

docker is already run in my system,you can see log as follows

2015-12-25 11:45:22,534[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 916] - Deploying web application archive /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war
2015-12-25 11:45:26,851[ost-startStop-1] [INFO ] [.c.p.d.c.DockerRegistryChecker 43] - Probing registry 'http://localhost:5000'
2015-12-25 11:45:26,871[ost-startStop-1] [INFO ] [.c.p.d.c.DockerRegistryChecker 48] - Probe of registry 'http://localhost:5000' succeed with HTTP response code '200'
2015-12-25 11:45:28,390[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 980] - Deployment of web application archive /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war has finished in 5,855 ms
2015-12-25 11:45:28,391[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 1036] - Deploying web application directory /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT
2015-12-25 11:45:28,429[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig 1152] - Deployment of web application directory /home/wu560130911/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT has finished in 38 ms
2015-12-25 11:45:28,433[main] [INFO ] [o.a.c.http11.Http11NioProtocol 469] - Starting ProtocolHandler ["http-nio-8080"]
2015-12-25 11:45:28,461[main] [INFO ] [o.a.catalina.startup.Catalina 638] - Server startup in 70781 ms
2015-12-25 11:46:17,857[8080-Acceptor-0] [ERROR] [o.a.t.util.net.NioEndpoint 709] - Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_65]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) ~[na:1.8.0_65]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) ~[na:1.8.0_65]
at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682) ~[tomcat-coyote.jar:8.0.28]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
2015-12-25 11:46:17,908[8080-Acceptor-0] [ERROR] [o.a.t.util.net.NioEndpoint 709] - Socket accept failed

@TylerJewell

This comment has been minimized.

Show comment
Hide comment
@TylerJewell

TylerJewell Dec 26, 2015

Contributor

I can see the access to the docker registry, but I didn't see the access to
docker itself.

We see the too many open files issue randomly. It often appears when using
docker through virtualbox & the virtualbox system has disk swapping or
memory constraint issue due to many reuses of the system.

Does "docker ps" work?
Have you tried installing virtualbox, launching docker in a virtualbox vm -
and do you get the same issues as running docker natively?

Tyler Jewell | CEO | tyler@​codenvy.​com | 9​78​.8​84​.53​55

On Fri, Dec 25, 2015 at 3:46 PM, 吴梦升 notifications@github.com wrote:

docker is already run in my system,you can see log as follows

2015-12-25 11:45:22,534[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
916] - Deploying web application archive
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war
2015-12-25 11:45:26,851[ost-startStop-1] [INFO ]
[.c.p.d.c.DockerRegistryChecker 43] - Probing registry '
http://localhost:5000'
2015-12-25 11:45:26,871[ost-startStop-1] [INFO ]
[.c.p.d.c.DockerRegistryChecker 48] - Probe of registry '
http://localhost:5000' succeed with HTTP response code '200'
2015-12-25 11:45:28,390[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
980] - Deployment of web application archive
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war
has finished in 5,855 ms
2015-12-25 11:45:28,391[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
1036] - Deploying web application directory
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT
2015-12-25 11:45:28,429[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
1152] - Deployment of web application directory
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT has
finished in 38 ms
2015-12-25 11:45:28,433[main] [INFO ] [o.a.c.http11.Http11NioProtocol 469]

  • Starting ProtocolHandler ["http-nio-8080"]
    2015-12-25 11:45:28,461[main] [INFO ] [o.a.catalina.startup.Catalina 638]
  • Server startup in 70781 ms
    2015-12-25 11:46:17,857[8080-Acceptor-0] [ERROR]
    [o.a.t.util.net.NioEndpoint 709] - Socket accept failed
    java.io.IOException: Too many open files
    at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_65]
    at
    sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
    ~[na:1.8.0_65]
    at
    sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
    ~[na:1.8.0_65]
    at
    org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682)
    ~[tomcat-coyote.jar:8.0.28]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
    2015-12-25 11:46:17,908[8080-Acceptor-0] [ERROR]
    [o.a.t.util.net.NioEndpoint 709] - Socket accept failed


Reply to this email directly or view it on GitHub
#140 (comment).

Contributor

TylerJewell commented Dec 26, 2015

I can see the access to the docker registry, but I didn't see the access to
docker itself.

We see the too many open files issue randomly. It often appears when using
docker through virtualbox & the virtualbox system has disk swapping or
memory constraint issue due to many reuses of the system.

Does "docker ps" work?
Have you tried installing virtualbox, launching docker in a virtualbox vm -
and do you get the same issues as running docker natively?

Tyler Jewell | CEO | tyler@​codenvy.​com | 9​78​.8​84​.53​55

On Fri, Dec 25, 2015 at 3:46 PM, 吴梦升 notifications@github.com wrote:

docker is already run in my system,you can see log as follows

2015-12-25 11:45:22,534[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
916] - Deploying web application archive
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war
2015-12-25 11:45:26,851[ost-startStop-1] [INFO ]
[.c.p.d.c.DockerRegistryChecker 43] - Probing registry '
http://localhost:5000'
2015-12-25 11:45:26,871[ost-startStop-1] [INFO ]
[.c.p.d.c.DockerRegistryChecker 48] - Probe of registry '
http://localhost:5000' succeed with HTTP response code '200'
2015-12-25 11:45:28,390[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
980] - Deployment of web application archive
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/che.war
has finished in 5,855 ms
2015-12-25 11:45:28,391[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
1036] - Deploying web application directory
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT
2015-12-25 11:45:28,429[ost-startStop-1] [INFO ] [o.a.c.startup.HostConfig
1152] - Deployment of web application directory
/home/hzwumsh/tools/eclipse-che-4.0.0-M6-SNAPSHOT/tomcat/webapps/ROOT has
finished in 38 ms
2015-12-25 11:45:28,433[main] [INFO ] [o.a.c.http11.Http11NioProtocol 469]

  • Starting ProtocolHandler ["http-nio-8080"]
    2015-12-25 11:45:28,461[main] [INFO ] [o.a.catalina.startup.Catalina 638]
  • Server startup in 70781 ms
    2015-12-25 11:46:17,857[8080-Acceptor-0] [ERROR]
    [o.a.t.util.net.NioEndpoint 709] - Socket accept failed
    java.io.IOException: Too many open files
    at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) ~[na:1.8.0_65]
    at
    sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
    ~[na:1.8.0_65]
    at
    sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
    ~[na:1.8.0_65]
    at
    org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682)
    ~[tomcat-coyote.jar:8.0.28]
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
    2015-12-25 11:46:17,908[8080-Acceptor-0] [ERROR]
    [o.a.t.util.net.NioEndpoint 709] - Socket accept failed


Reply to this email directly or view it on GitHub
#140 (comment).

@eivantsov

This comment has been minimized.

Show comment
Hide comment
@eivantsov

eivantsov Dec 26, 2015

2015-12-25 11:46:17,857[8080-Acceptor-0] [ERROR] [o.a.t.util.net.NioEndpoint 709] - Socket accept failed java.io.IOException: Too many open files actually means there's no connection to Docker socket. Make sure you can run docker ps in the terminal where you launch Che.

Info on your system setup will also help.

eivantsov commented Dec 26, 2015

2015-12-25 11:46:17,857[8080-Acceptor-0] [ERROR] [o.a.t.util.net.NioEndpoint 709] - Socket accept failed java.io.IOException: Too many open files actually means there's no connection to Docker socket. Make sure you can run docker ps in the terminal where you launch Che.

Info on your system setup will also help.

@wu560130911

This comment has been minimized.

Show comment
Hide comment
@wu560130911

wu560130911 Dec 28, 2015

root@mini02-04:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e26315acde08 dockerui/dockerui "/dockerui" 44 hours ago Up 44 hours 0.0.0.0:9000->9000/tcp romantic_spence
e218a1eb2041 registry "docker-registry" 44 hours ago Up 44 hours 0.0.0.0:5000->5000/tcp registry

wu560130911 commented Dec 28, 2015

root@mini02-04:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e26315acde08 dockerui/dockerui "/dockerui" 44 hours ago Up 44 hours 0.0.0.0:9000->9000/tcp romantic_spence
e218a1eb2041 registry "docker-registry" 44 hours ago Up 44 hours 0.0.0.0:5000->5000/tcp registry

@eivantsov

This comment has been minimized.

Show comment
Hide comment
@eivantsov

eivantsov Mar 3, 2016

Startup script now does a series of checks, including Docker availability.

Closing the issue.

eivantsov commented Mar 3, 2016

Startup script now does a series of checks, including Docker availability.

Closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment