New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

日志系统增强 #1

Open
2d0g opened this Issue Jun 27, 2017 · 0 comments

Comments

Projects
None yet
1 participant
@2d0g
Owner

2d0g commented Jun 27, 2017

1.引入 request id 快速定位请求链

image

场景: 某一次请求,A系统调用B系统,B系统调用C系统。A产生了异常如何快速通过ELK日志系统定位该次A,B,C系统请求链。
方案: 使用X-Request-ID Header。让web入口 nginx 生成X-Request-ID Header 各个web系统(应用服务器+nginx等)传递输出X-Reqeust-ID。
难点: 如何让nginx产生X-Request-ID,并且产生的逻辑是如果请求已经有该Header则不生成,没有该Header则生成。并在nginx和app之间传递和输出。

nginx v1.10后 引入$request_id变量,由于我这边使用了最新版本的openresty,所以nginx可以直接使用该变量。具体nginx配置如下:

http {
        map $host $rid {
                default '';
        }
        log_format main '.... "$rid"';
}

server {

        ......

        set $rid $request_id;
        if ($http_x_request_id != '') {
                set $rid $http_x_request_id;
        }
        proxy_set_header X-Request-ID $rid;
        
        location / {
                  ......
        }
        ......
}

而应用服务器的输出,一般框架都可以通过配置中间件统一对日志加字段,可以参考
https://devcenter.heroku.com/articles/http-request-id
此外,框架的http client再调用其他api的时候也必需传递该header

image

反馈waf规则

日志监控与控制台增加

  1. x-pack 及相关插件, 此外还有
  2. 类似 tail 的功能 logtril
  3. es 数据轮转 curator

@2d0g 2d0g added bug ansible devops java and removed bug ansible java labels Jul 4, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment