Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

could not build the variables_hash on 1.4.1 #21

Closed
dctrwatson opened this Issue July 29, 2013 · 7 comments

2 participants

John Watson Yichun Zhang
John Watson

After upgrading from 1.2.9 to 1.4.1 receive this on startup:

[emerg] could not build the variables_hash, you should increase either variables_hash_max_size: 512 or variables_hash_bucket_size: 64

Setting variables_hash_max_size: 1024 fixes the issue, but this wasn't required for 1.2.9

Using tag v0.21

Yichun Zhang
Owner

In both nginx 1.2.9 and nginx 1.4.1, the default value of variables_hash_max_size is 512.

Will you provide a minimal nginx configuration that can reproduce this emerg message with nginx 1.4.1?

Thanks!

Yichun Zhang
Owner

Okay, checking out your nginx.conf file here: https://gist.github.com/dctrwatson/6075317

I cannot reproduce the emerg error message here with nginx 1.4.1 + ngx_srcache 0.21 + your nginx.conf. I build nginx this way:

./configure --prefix=/opt/nginx --with-http_realip_module \
        --with-http_stub_status_module \
        --add-module=$HOME/git/srcache-nginx-module

My hunch is that it could be that nginx 1.4.1 adds a lot more variables and you were also enabling quite a few modules which happens to exceed the default hash table limit. The limit was not hit with nginx 1.2.9 on your side just because nginx 1.2.9 had fewer built-in variables.

So increasing variables_hash_max_size in your nginx.conf is the right thing to do.

Regards,
-agentzh

John Watson

Sorry, I guess I didn't debug this very thoroughly.

I went through a bunch of combinations of the 3rd party modules listed here: https://gist.github.com/dctrwatson/6075317#file-nginx-1-4-1-v

Though I had always kept:

--add-module=/home/vagrant/nginx/debian/modules/nginx-development-kit
--add-module=/home/vagrant/nginx/debian/modules/set-misc-nginx-module
--add-module=/home/vagrant/nginx/debian/modules/headers-more-nginx-module
--add-module=/home/vagrant/nginx/debian/modules/nginx-echo

I just ran some more permutations and narrowed it down to this combination (in either order):

--add-module=/home/vagrant/nginx/debian/modules/nginx-echo
--add-module=/home/vagrant/nginx/debian/modules/srcache-nginx-module

Yichun Zhang
Owner

Still cannot reproduce the error on my side with nginx 1.4.1 + ngx_echo v0.45 + ngx_srcache v0.21:

$ /opt/nginx/sbin/nginx -V
nginx version: nginx/1.4.1
built by gcc 4.7.2 20120921 (Red Hat 4.7.2-2) (GCC) 
configure arguments: --prefix=/opt/nginx --with-http_realip_module --prefix=/opt/nginx --with-http_stub_status_module --add-module=/home/agentzh/git/echo-nginx-module --add-module=/home/agentzh/git/srcache-nginx-module
Yichun Zhang
Owner

Okay, I can finally reproduce that [emerg] message on my side by also enabling all those extra nginx std modules:

--with-http_addition_module \
--with-http_geoip_module \
--with-http_gzip_static_module \
--with-http_image_filter_module \
--with-http_realip_module \
--with-http_spdy_module \
--with-http_stub_status_module \
--with-http_ssl_module \
--with-http_sub_module \
--with-http_xslt_module \

And analyzing the backtrace of the position that emits the [emerg] message, I can see that it's indeed caused by just too many registered built-in nginx variables:

(gdb) bt
#0  ngx_hash_init (hinit=hinit@entry=0x7fffffffdbf0, names=0x707330, nelts=124) at src/core/ngx_hash.c:315
#1  0x0000000000448441 in ngx_http_variables_init_vars (cf=cf@entry=0x7fffffffe1a0) at src/http/ngx_http_variables.c:2532
#2  0x000000000042fee0 in ngx_http_block (cf=0x7fffffffe1a0, cmd=<optimized out>, conf=<optimized out>)
  at src/http/ngx_http.c:316
#3  0x000000000041812f in ngx_conf_handler (last=1, cf=0x7fffffffe1a0) at src/core/ngx_conf_file.c:387
#4  ngx_conf_parse (cf=cf@entry=0x7fffffffe1a0, filename=filename@entry=0x6f4020) at src/core/ngx_conf_file.c:243
#5  0x0000000000415b86 in ngx_init_cycle (old_cycle=old_cycle@entry=0x7fffffffe240) at src/core/ngx_cycle.c:268
#6  0x000000000040833c in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:333

To conclude, this has nothing to do with ngx_srcache nor ngx_echo. You just have a lot of nginx built-in variables registered by both the nginx std modules and 3rd-party modules. And there's no bugs here that I can see.

Maybe the nginx core can increase the default value of variables_hash_max_size directive. 512 is just not really big enough for the rich ecosystem nowadays :)

John Watson

It just seemed weird the error occurs with 2 less 3rd party modules than we had in 1.2.9 and ngx_echo and ngx_srcache are mutually exclusive in 1.4.1 (I can run with no issues, as long as only 1 of those 2 modules is installed.) Also, that openresty 1.4.1.1 didn't run into this issue with so many more modules installed.

Thanks for your help looking into this!

I'll just leave variables_hash_max_size: 1024 in the configuration since like you said, this isn't a bug with ngx_srcache or ngx_echo.

Yichun Zhang
Owner

Yeah, when you add more modules you usually add more built-in nginx variables (because modules can register its own built-in nginx variables). So it's natural to hit the variables_hash_max_size limit when you add more and more modules anyway.

Because there's no bugs in ngx_srcache. I'm closing this.

Yichun Zhang agentzh closed this July 30, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.