Skip to content

Compressed Log sink for Rust's log crate with IoT in mind

License

Notifications You must be signed in to change notification settings

mpapierski/compressed_log

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

51 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

compressed_log

Latest Version Documentation

A Rust crate to compress logs on the fly, and send them over the network after reaching a configured threshold.

This is useful for when you have a fleet of embedded devices running a Rust program, instead of building a complicated metrics framework you can collect the normal Rust log output securely using https and efficiently using LZ4 compression.

LZ4 is both fast enough to run on even the most resource starved devices and effective enough to reduce log size by 20x or more from naive byte streams.

On the server side logs are simply dumped into a file, where they can be aggregated and processed by standard log collection tools.

Features

  • Uses log API
  • LZ4 to compress log on the fly
  • Transmits compressed data over persistent WebSocket connection
  • Configurable threshold which will trigger in-memory log rotation and data transmission.
  • Fully architecture portable, including Mips and other BE architectures

Example

Check out examples folder for more examples. Including custom formats for collected logs.

Client:

let level = Level::Info;
let logger = LoggerBuilder::new()
    .set_level(level)
    .set_compression_level(Compression::Slow)
    .set_sink_url("http://127.0.0.1:8000/sink/")
    .build()?;
log::set_boxed_logger(Box::new(logger))?;
log::set_max_level(level.to_level_filter());

Server:

The current server reference implementaiton is compressed_log_sink

About

Compressed Log sink for Rust's log crate with IoT in mind

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages