Usage with stream module

[SalmaanP]

Hi, I'm having some trouble seeing metrics with the stream module. Not sure what is missing since I don't see any errors.
Here's the conf ( there is a different server and upstream block not shown here for simplicity)

stream {
  lua_shared_dict prometheus_metrics 10M;
  lua_package_path "/tmp/nginx-lua-prometheus/?.lua;/usr/local/openresty/luajit/lib/lua/5.1/?.lua;;";
  init_by_lua '
    prometheus = require("prometheus").init("prometheus_metrics")
    metric_requests = prometheus:counter("nginx_http_requests_total", "Number of HTTP requests", {"test"})
    metric_connections = prometheus:gauge("nginx_http_connections", "Number of HTTP connections", {"test"})
 ';
  log_by_lua_block {
    metric_connections:set(1, {"test"})
 } 

 server {
  listen 9145;
  content_by_lua '
    local sock = assert(ngx.req.socket(true))
    local data = sock:receive()
    local location = "GET /metrics"
    if string.sub(data, 1, string.len(location)) == location then
      ngx.say("HTTP/1.1 200 OK")
      ngx.say("Content-Type: text/plain")
      ngx.say(table.concat(prometheus:metric_data(), ""))
    else
      ngx.say("HTTP/1.1 404 Not Found")
    end
  ';
  }

When hitting the endpoint, I don't see the metrics_requests or metrics_connections metrics but only the default error metric which is in the response header and not body.

curl localhost:9145/metrics -v
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9145 (#0)
> GET /metrics HTTP/1.1
> Host: localhost:9145
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain
< # HELP nginx_metric_errors_total Number of nginx-lua-prometheus errors
< # TYPE nginx_metric_errors_total counter
< nginx_metric_errors_total 0
* no chunk, no close, no size. Assume close to signal end
<
* Curl_http_done: called premature == 0
* Closing connection 0

[knyar]

Note that you are not actually incrementing the metrics_requests metric, so you should not expect to see it in the output. However, I would expect to see a value of 1 for metric_connections (assuming the server got at least one request before you collected metrics).

I am not really familiar with the stream module. Are you sure that it is fully supported by the lua module, with log_by_lua_block getting executed?

[SalmaanP]

yup, the log_by_lua_block is not getting executed. Moving the metric_connections:set(1, {"test"}) to the init block shows the metric. (again in headers which is weird). I will just workaround with moving everything to the http block and doing this instead.

  log_by_lua_block {
    metric_connections:set(ngx.var.connections_active, {"active"})
    metric_connections:set(ngx.var.connections_reading, {"reading"})
    metric_connections:set(ngx.var.connections_writing, {"writing"})
    metric_connections:set(ngx.var.connections_waiting, {"waiting"})
    metric_requests:inc(1, {"test"})
  }

[naveenbharadwaj-nc]

I'm trying to proxy requests based on a port number in nginx. So I have multiple server blocks under stream module that proxies data by listening on various ports. I want to track each requests by adding it to metrics. So I have used log_by_lua_block inside each server block and using similar code from previous comment. But I dont get metrics except default error one.
My stream module looks like this:

stream {
    log_format proxy '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received ';
    access_log /usr/local/openresty/nginx/tcp_access.log proxy;
    lua_shared_dict prometheus_metrics 10M;
    lua_package_path '/usr/local/openresty/lualib/?.lua;;';
    lua_code_cache on;

    init_worker_by_lua_block {
        prometheus = require("prometheus").init("prometheus_metrics")
        metric_requests = prometheus:counter(
          "nc_diagnostic_agent_http_requests_total", "Number of HTTP requests", {"host", "status"})
        metric_latency = prometheus:histogram(
          "nc_diagnostic_agent_http_request_duration_seconds", "HTTP request latency", {"host"})
        metric_connections = prometheus:gauge("nc_diagnostic_agent_http_connections", "Number of HTTP connections", {"host"})    
    }

    server {
        listen 9999;
        content_by_lua_block {
            local sock = assert(ngx.req.socket(true))
            local data = sock:receive()
            local location = "GET /metrics"
            if string.sub(data, 1, string.len(location)) == location then
                ngx.say("HTTP/1.1 200 OK")
                ngx.say("Content-Type: text/plain")
                ngx.say("")
                ngx.say(table.concat(prometheus:metric_data(), ""))
            else
                ngx.say("HTTP/1.1 404 Not Found")
            end
        }
    }
resolver 172.30.0.10 valid=10s;

    server {
      listen 0.0.0.0:1715;
      log_by_lua_block {
        local host = ngx.var.host:gsub("^www.", "")
        metric_requests:inc(1, {host, ngx.var.status})
        metric_latency:observe(ngx.now() - ngx.req.start_time(), {host})
        metric_connections:set(ngx.var.connections_active, {"active"})
        metric_connections:set(ngx.var.connections_reading, {"reading"})
        metric_connections:set(ngx.var.connections_writing, {"writing"})
        metric_connections:set(ngx.var.connections_waiting, {"waiting"})                  
      }
      proxy_pass test.com:1715;
      proxy_buffer_size 8k;
      proxy_connect_timeout 1s;
    }
    server {
      listen 0.0.0.0:8080;
      log_by_lua_block {
        local host = ngx.var.host:gsub("^www.", "")
        metric_requests:inc(1, {host, ngx.var.status})
        metric_latency:observe(ngx.now() - ngx.req.start_time(), {host})
        metric_connections:set(ngx.var.connections_active, {"active"})
        metric_connections:set(ngx.var.connections_reading, {"reading"})
        metric_connections:set(ngx.var.connections_writing, {"writing"})
        metric_connections:set(ngx.var.connections_waiting, {"waiting"})                  
      }
      proxy_pass test.com:8080;
      proxy_buffer_size 8k;
      proxy_connect_timeout 1s;
    }
   .....
   //some more server blocks that listen to different port numbers with same logic.
}

Can somebody please point out my mistake?