Log Trace

分布式日志追踪和收集系统 🛠 | Distributed log tracking and collection system

介绍😊

Log Trace分为客户端和服务端，客户端收集日志，服务端存储日志；客户端项目依赖logback等日志框架，依赖springboot环境

目录📚

|-- README.md
|-- log-trace-demo # 示例工程
|-- log-trace-server # 服务端
|-- log-trace-spring-boot-starter # 客户端
`-- pom.xml

执行流程🎡

sequenceDiagram
Appender->Queue: Client：日志框架收集日志到RedisAppender，具体数据存到本地队列
Queue-->>Redis: Client：从本地队列批量取出数据，缓存到Redis队列
Redis->ElasticSearch: Server：取出数据，存至ES

Loading

目前仅适配logback的appender

    <appender name="LOG_TRACE" class="com.log.trace.appender.RedisAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
    </appender>

使用方法📖

在需要集成的业务项目下

业务项目pom文件引入客户端

    <dependency>
        <groupId>com.log.trace</groupId>
        <artifactId>log-trace-spring-boot-starter</artifactId>
        <version>0.0.1</version>
    </dependency>

业务项目yaml配置

yaml文件增加log-trace使用的redis连接

注意：此配置不影响spring-boot-starter-data-redis的autoconfig

## log-trace-client配置
trace:
  redis:
    host: # your redis ip
    port: # your redis port
    password:
    database: 15 # 默认15号库
    pool:
      #最小空闲连接数
      min-idle: 1
      #最大空闲连接数
      max-idle: 2
      #最大数据库连接数
      max-active: 2
      #连接池最大阻塞等待时间
      max-wait: 1000ms

业务项目logback配置

    <appender name="LOG_TRACE" class="com.log.trace.appender.RedisAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
    </appender>
        
    <!-- 注意这个name，固定为TRACE -->
    <logger name="TRACE" level="INFO">
        <appender-ref ref="LOG_TRACE"/>
    </logger>

业务项目配置自己的Redis

建议如下配置：

@Configuration
@AutoConfigureAfter(RedisAutoConfiguration.class)
public class RedisConfig {
    
    @Primary
    @Bean(name = "lettuceConnectionFactory")
    public LettuceConnectionFactory defaultRedisConnectionFactory(RedisStandaloneConfiguration redisStandaloneConfiguration) {
        return new LettuceConnectionFactory(redisStandaloneConfiguration);
    }
    
    @Primary
    @Bean(name = "redisStandaloneConfiguration")
    public RedisStandaloneConfiguration redisConfiguration(RedisProperties redisProperties){
        RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration();
        configuration.setHostName(redisProperties.getHost());
        configuration.setPassword(redisProperties.getPassword());
        configuration.setDatabase(redisProperties.getDatabase());
        configuration.setPort(redisProperties.getPort());
        return configuration;
    }
    
    @Bean
    public RedisTemplate<Object, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory){
        RedisTemplate<Object, Object> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(lettuceConnectionFactory);
        // 使用Jackson2JsonRedisSerialize 替换默认的jdkSerializeable序列化
        Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
        ObjectMapper objectMapper = new ObjectMapper().disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        jackson2JsonRedisSerializer.setObjectMapper(objectMapper);
        // 设置value的序列化规则和 key的序列化规则
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }
}

业务项目使用

LOG.info("业务操作日志信息 info:{} | Business operation log information info:{}",dateTimeStr,dateTimeStr);

服务端修改

服务端需要修改yaml文件的spring.redis.端口、IP等，保持和客户端的一致

其他🩺

1.与OpenFeign项目配合

与OpenFeign项目配合使用时，需要加入拦截器，传递traceId

@Component
public class FeignInterceptor implements RequestInterceptor {

    public static final String TRACE_ID_HEADER = "traceId";

    @Override
    public void apply(RequestTemplate requestTemplate) {
        requestTemplate.header(TRACE_ID_HEADER, MDC.get(TRACE_ID_HEADER));
    }
}

2.与SpringCloudGateway项目配合

与SpringCloudGateway项目配合使用时，需要加入过滤器，初始化traceId

@Component
public class TraceFilter implements GlobalFilter, Ordered {

    public static final String TRACE_ID_HEADER = "traceId";
    
    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        exchange.getRequest().mutate().header(TRACE_ID_HEADER, IdUtil.randomUUID()).build();
        return chain.filter(exchange);
    }

    @Override
    public int getOrder() {
        return 0;
    }
}

3.其他Http调用工具配合

使用RestTemplate,OkHttp,HttpClient等工具时，如果需要传递traceId，请添加如下拦截器到工具中

com.log.trace.global.interceptor.HttpClientTraceIdInterceptor
com.log.trace.global.interceptor.OkHttpTraceIdInterceptor
com.log.trace.global.interceptor.RestTemplateTraceIdInterceptor

例如：

    @Bean
    public RestTemplate restTemplate(ClientHttpRequestFactory factory){
        RestTemplate restTemplate = new RestTemplate(factory);
        restTemplate.setInterceptors(Collections.singletonList(new RestTemplateTraceIdInterceptor()));
        return restTemplate;
    }

详细请看log-trace-demo工程