The value of total connections is wrong #35
Labels
Comments
|
Hi, I’d like to ask you some questions.
|
|
OH, my god, it maybe a serious problem, looks like a shared memory operation error:
The I also find a error log below: Looks like it happened when I restarted Nginx Last week. |
|
The abnormal values probably seem to happen because of the failure to decrease the status variable values. In other words, It seems to due to the terminating of worker process by signal 11 before running decrease process. By the way, more important thing is why worker process exited on signal 11. Status variablesThe status variables used in ngx_http_stub_status_module and nginx-module-vts are a shared memory variables(atomic). Some variables are only increased without decrease as follows:
Increase function (void) ngx_atomic_fetch_add($ngx_stat_variable, 1);Decrease function (void) ngx_atomic_fetch_add($ngx_stat_variable, -1);Some questions
backtraceCheck max core file sizeSee the nginx debugging.
Script to check max core file size
#! /bin/env bash
echo "# Max core file size of nginx worker process for core dump(must be soft > 0)"
printf "%-16s%-16s%-16s\n" "PID" "Soft Limit" "Hard Limit"
for pid in $(echo /proc/[0-9]*)
do
[ -e "$pid/cmdline" ] && cmdline=$(<$pid/cmdline)
if [[ $cmdline == *"nginx: worker"* ]]
then
limits=$(<$pid/limits)
core_soft_limit=$(awk '/core/ {print $5" "$6}' $pid/limits)
printf "%-16s%-16s%-16s\n" $pid $core_soft_limit
fi
done
# Max core file size of nginx worker process for core dump(must be soft > 0)
PID Soft Limit Hard Limit
/proc/24262 unlimited unlimited
/proc/24263 unlimited unlimited Source traceTraced the nginx core source(1.9.12) The signal 11 is as follows:Trace the message
|
|
I have added the |
|
Hi, I have got the |
|
Thanks. Source traceTrace ngx_http_complex_value_t member variable flushes
35 void
36 ngx_http_script_flush_complex_value(ngx_http_request_t *r,
37 ngx_http_complex_value_t *val)
38 {
39 ngx_uint_t *index;
40
41 index = val->flushes;
42
43 if (index) {
44 while (*index != (ngx_uint_t) -1) {
45
46 if (r->variables[*index].no_cacheable) {
47 r->variables[*index].valid = 0;
48 r->variables[*index].not_found = 0;
49 }
50
51 index++;
52 }
53 }
54 }
.
.
107 ngx_int_t
108 ngx_http_compile_complex_value(ngx_http_compile_complex_value_t *ccv)
109 {
.
.
117 nv = 0;
118 nc = 0;
119
120 for (i = 0; i < v->len; i++) {
121 if (v->data[i] == '$') {
122 if (v->data[i + 1] >= '1' && v->data[i + 1] <= '9') {
123 nc++;
124
125 } else {
126 nv++;
127 }
128 }
129 }
130
.
.
141
142 ccv->complex_value->value = *v;
143 ccv->complex_value->flushes = NULL;
144 ccv->complex_value->lengths = NULL;
145 ccv->complex_value->values = NULL;
146
147 if (nv == 0 && nc == 0) {
148 return NGX_OK;
149 }
.
. |
|
It has been working fine for a long time since you fixed the bug. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, I'm back
The new problem is that the value of total connections in
Server mainof the status webpage is too big, it's wrong, it should be less than 15k, there's not so many connections in this one of our systems. after I restart Nginx, it's back to normal(about 10000 connections), but after about a week, it's into wrong again.BTW, when it's in the abnormal state, the data of original module
stub_status_moduleof Nginx is also abnormal, of course, they are equal.SS:
nginx-module-vts:
stub_status_module:
The text was updated successfully, but these errors were encountered: