Skip to content

Latest commit

 

History

History
427 lines (308 loc) · 12.7 KB

errlog.md

File metadata and controls

427 lines (308 loc) · 12.7 KB
Raw

Name

ngx.errlog - manage nginx error log data in Lua for OpenResty/ngx_lua.

Table of Contents

Status

This Lua module is production ready.

Synopsis

Capturing nginx error logs with specified log filtering level

error_log logs/error.log info;

http {
    # enable capturing error logs
    lua_capture_error_log 32m;

    init_by_lua_block {
        local errlog = require "ngx.errlog"
        local status, err = errlog.set_filter_level(ngx.WARN)
        if not status then
            ngx.log(ngx.ERR, err)
            return
        end
        ngx.log(ngx.WARN, "set error filter level: WARN")
    }

    server {
        # ...
        location = /t {
            content_by_lua_block {
                local errlog = require "ngx.errlog"
                ngx.log(ngx.INFO, "test1")
                ngx.log(ngx.WARN, "test2")
                ngx.log(ngx.ERR, "test3")

                local logs, err = errlog.get_logs(10)
                if not logs then
                    ngx.say("FAILED ", err)
                    return
                end

                for i = 1, #logs, 3 do
                    ngx.say("level: ", logs[i], " time: ", logs[i + 1],
                            " data: ", logs[i + 2])
                end
            }
        }
    }
}

The example location above produces a response like this:

level: 5 time: 1498546995.304 data: 2017/06/27 15:03:15 [warn] 46877#0:
    [lua] init_by_lua:8: set error filter level: WARN
level: 5 time: 1498546999.178 data: 2017/06/27 15:03:19 [warn] 46879#0: *1
    [lua] test.lua:5: test2, client: 127.0.0.1, server: localhost, ......
level: 4 time: 1498546999.178 data: 2017/06/27 15:03:19 [error] 46879#0: *1
    [lua] test.lua:6: test3, client: 127.0.0.1, server: localhost, ......

Back to TOC

Methods

set_filter_level

syntax: status, err = log_module.set_filter_level(log_level)

context: init_by_lua*

Specifies the filter log level, only to capture and buffer the error logs with a log level no lower than the specified level.

If we don't call this API, all of the error logs will be captured by default.

In case of error, nil will be returned as well as a string describing the error.

This API should always work with directive lua_capture_error_log.

See Nginx log level constants for all nginx log levels.

For example,

 init_by_lua_block {
     local errlog = require "ngx.errlog"
     errlog.set_filter_level(ngx.WARN)
 }

NOTE: The debugging logs since when OpenResty or NGINX is not built with --with-debug, all the debug level logs are suppressed regardless.

Back to TOC

get_logs

syntax: res, err = log_module.get_logs(max?, res?)

context: any

Fetches the captured nginx error log messages if any in the global data buffer specified by ngx_lua's lua_capture_error_log directive. Upon return, this Lua function also removes those messages from that global capturing buffer to make room for future new error log data.

In case of error, nil will be returned as well as a string describing the error.

The optional max argument is a number that when specified, will prevent errlog.get_logs from adding more than max messages to the res array.

for i = 1, 20 do
   ngx.log(ngx.ERR, "test")
end

local errlog = require "ngx.errlog"
local res = errlog.get_logs(10)
-- the number of messages in the `res` table is 10 and the `res` table
-- has 30 elements.

The resulting table has the following structure:

{ level1, time1, msg1, level2, time2, msg2, ... }

The levelX values are constants defined below:

https://github.com/openresty/lua-nginx-module/#nginx-log-level-constants

The timeX values are UNIX timestamps in seconds with millisecond precision. The sub-second part is presented as the decimal part. The time format is exactly the same as the value returned by ngx.now. It is also subject to NGINX core's time caching.

The msgX values are the error log message texts.

So to traverse this array, the user can use a loop like this:

for i = 1, #res, 3 do
    local level = res[i]
    if not level then
        break
    end

    local time = res[i + 1]
    local msg = res[i + 2]

    -- handle the current message with log level in `level`,
    -- log time in `time`, and log message body in `msg`.
end

Specifying max <= 0 disables this behavior, meaning that the number of results won't be limited.

The optional 2th argument res can be a user-supplied Lua table to hold the result instead of creating a brand new table. This can avoid unnecessary table dynamic allocations on hot Lua code paths. It is used like this:

local errlog = require "ngx.errlog"
local new_tab = require "table.new"

local buffer = new_tab(100 * 3, 0)  -- for 100 messages

local errlog = require "ngx.errlog"
local res, err = errlog.get_logs(0, buffer)
if res then
    -- res is the same table as `buffer`
    for i = 1, #res, 3 do
        local level = res[i]
        if not level then
            break
        end
        local time = res[i + 1]
        local msg  = res[i + 2]
        ...
    end
end

When provided with a res table, errlog.get_logs won't clear the table for performance reasons, but will rather insert a trailing nil value after the last table element.

When the trailing nil is not enough for your purpose, you should clear the table yourself before feeding it into the errlog.get_logs function.

Back to TOC

get_sys_filter_level

syntax: log_level = log_module.get_sys_filter_level()

context: any

Return the nginx core's error log filter level (defined via the error_log configuration directive in nginx.conf) as an integer value matching the nginx error log level constants documented below:

https://github.com/openresty/lua-nginx-module/#nginx-log-level-constants

For example:

local errlog = require "ngx.errlog"
local log_level = errlog.get_sys_filter_level()
-- Now the filter level is always one level higher than system default log level on priority
local status, err = errlog.set_filter_level(log_level - 1)
if not status then
    ngx.log(ngx.ERR, err)
    return
end

Back to TOC

raw_log

syntax: log_module.raw_log(log_level, msg)

context: any

Log msg to the error logs with the given logging level.

Just like the ngx.log API, the log_level argument can take constants like ngx.ERR and ngx.WARN. Check out Nginx log level constants for details.

However, unlike the ngx.log API which accepts variadic arguments, this function only accepts a single string as its second argument msg.

This function differs from ngx.log in the way that it will not prefix the written logs with any sort of debug information (such as the caller's file and line number).

For example, while ngx.log would produce

2017/07/09 19:36:25 [notice] 25932#0: *1 [lua] content_by_lua(nginx.conf:51):5: hello world, client: 127.0.0.1, server: localhost, request: "GET /log HTTP/1.1", host: "localhost"

from

ngx.log(ngx.NOTICE, "hello world")

the errlog.raw_log() call produces

2017/07/09 19:36:25 [notice] 25932#0: *1 hello world, client: 127.0.0.1, server: localhost, request: "GET /log HTTP/1.1", host: "localhost"

from

local errlog = require "ngx.errlog"
errlog.raw_log(ngx.NOTICE, "hello world")

This function is best suited when the format and/or stack level of the debug information proposed by ngx.log is not desired. A good example of this would be a custom logging function which prefixes each log with a namespace in an application:

1.  local function my_log(lvl, ...)
2.      ngx.log(lvl, "[prefix] ", ...)
3.  end
4.
5.  my_log(ngx.ERR, "error")

Here, the produced log would indicate that this error was logged at line 2., when in reality, we wish the investigator of that log to realize it was logged at line 5. right away.

For such use cases (or other formatting reasons), one may use raw_log to create a logging utility that supports such requirements. Here is a suggested implementation:

local errlog = require "ngx.errlog"

local function my_log(lvl, ...)
  -- log to error logs with our custom prefix, stack level
  -- and separator
  local n = select("#", ...)
  local t = { ... }
  local info = debug.getinfo(2, "Sl")

  local prefix = string.format("(%s):%d:", info.short_src, info.currentline)
  local buf = { prefix }

  for i = 1, n do
    buf[i + 1] = tostring(t[i])
  end

  local msg = table.concat(buf, " ")

  errlog.raw_log(lvl, msg) -- line 19.
end

local function my_function()
  -- do something and log

  my_log(ngx.ERR, "hello from", "raw_log:", true) -- line 25.
end

my_function()

This utility function will produce the following log, explicitly stating that the error was logged on line 25.:

2017/07/09 20:03:07 [error] 26795#0: *2 (/path/to/file.lua):25: hello from raw_log: true, context: ngx.timer

As a reminder to the reader, one must be wary of the cost of string concatenation on the Lua land, and should prefer the combined use of a buffer table and table.concat to avoid unnecessary GC pressure.

Back to TOC

Community

Back to TOC

English Mailing List

The openresty-en mailing list is for English speakers.

Back to TOC

Chinese Mailing List

The openresty mailing list is for Chinese speakers.

Back to TOC

Bugs and Patches

Please report bugs or submit patches by

  1. creating a ticket on the GitHub Issue Tracker,
  2. or posting to the OpenResty community.

Back to TOC

Author

Yuansheng Wang <membphis@gmail.com> (membphis), OpenResty Inc.

Back to TOC

Copyright and License

This module is licensed under the BSD license.

Copyright (C) 2017, by Yichun "agentzh" Zhang, OpenResty Inc.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Back to TOC

See Also

Back to TOC