There are too many "TIME_WAIT" between nginx and tomcat when your ajp module is in use.

[HelloJamesLee]

Hi,

When I use Nginx+ajp_module+tomcat and Apache+mod_jk+tomcat, I found two problems. What's the reason? Can you help?

(1) There are too many TIME_WAIT between nginx and tomcat
The TIME_WAIT/Total connection is 13890/16030
When I use the Apache+mod_jk+tomcat, the TIME_WAIT/Total is 92/952.

(2) The %CPU of tomcat in Nginx is higher than that of tomcat in apache. -- This is the main problem.
The tomcat in Apache used all the maxThreads(512), but tomcat in Nginx only uses 125 threads, via "ps -efL | grep catalina "

The configuration of tomcats in these two situatin are the same, and the concurrency is also the same. The configuration of ajp module is as follow:
ajp_connect_timeout 10;
ajp_read_timeout 10;
upstream loadbalancer {
server 127.0.0.1:8001;
server 127.0.0.1:8002;
server 127.0.0.1:8003;

    #keepalive 6400;
}

Whether i enable keepalive in upstream directive or not, the two problem exists all the same.
The ajp_module version is 0.2.5, the latest version.

Can you give me some clues to reslove these problems? Thank you!

[yaoweibin]

What's your Nginx version?

On 2011-11-4 13:17, HelloJamesLee wrote:

Hi,

When I use Nginx+ajp_module+tomcat and Apache+mod_jk+tomcat, I found two problems. What's the reason? Can you help?

(1) There are too many TIME_WAIT between nginx and tomcat
The TIME_WAIT/Total connection is 13890/16030
When I use the Apache+mod_jk+tomcat, the TIME_WAIT/Total is 92/952.

(2) The %CPU of tomcat in Nginx is higher than that of tomcat in apache. -- This is the main problem.
The tomcat in Apache used all the maxThreads(512), but tomcat in Nginx only uses 125 threads, via "ps -efL | grep catalina "

The configuration of tomcats in these two situatin are the same, and the concurrency is also the same. The configuration of ajp module is as follow:
ajp_connect_timeout 10;
ajp_read_timeout 10;
upstream loadbalancer {
server 127.0.0.1:8001;
server 127.0.0.1:8002;
server 127.0.0.1:8003;

     #keepalive 6400;
 }

Whether i enable keepalive in upstream directive or not, the two problem exists all the same.
The ajp_module version is 0.2.5, the latest version.

Can you give me some clues to reslove these problems? Thank you!

---

Reply to this email directly or view it on GitHub:
#5

Weibin Yao

[HelloJamesLee]

My Nginx is 0.8.54.

There is the connection pool mechanism in mod_jk of apache. The connection_pool_size in mod_jk is automatically detected by mod_jk according to the number of threads per web server process. The connections between apache and tomcat can be reused by mod_jk. So the TIME_WAIT in apache is lower.

I guess that the keepalive between nginx and tomcat dosen't work very well. Only a small number connections can be keeped alive. Do you think so?

[yaoweibin]

Can you show me the debug.log with several requests?

http://wiki.nginx.org/Debugging

[wangbin579]

try to set accept_mutex off
maybe this will help you

[HelloJamesLee]

The reason:
In the test servers, there are many times the read event was triggered. But the socket didn't read any data.
In the old keepalive module, it just closes the idle keepalive socket. It may cause much unexpected connection closing.

Weibin has fixed this keepalive problem.
Thanks Weibin! Thank you!