Skip to content

Java library to send structured logs (JSONs) to Logz.io via HTTPS. This the core component of Logzio Appenders in the different logging frameworks (log4j, log4j2, Logback) which can also be used on its own

master
Switch branches/tags
Go to file
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Build Status Maven Central

Logzio java sender

This sender sends logs messages to your Logz.io account in JSON format, using non-blocking threading, bulks, and HTTPS encryption. Please note that this sender requires java 8 and up.

Technical Information

This appender uses BigQueue implementation of persistent queue, so all logs are backed up to a local file system before being sent. Once you send a log, it will be enqueued in the queue and 100% non-blocking. There is a background task that will handle the log shipment for you. This jar is an "Uber-Jar" that shades both BigQueue, Gson and Guava to avoid "dependency hell".

Installation from maven

<dependency>
    <groupId>io.logz.sender</groupId>
    <artifactId>logzio-sender</artifactId>
    <version>${logzio-sender-version}</version>
</dependency>

Parameters

Parameter Default Explained
token None Your Logz.io token, which can be found under "settings" in your account.
logzioType java The log type for that sender
drainTimeoutSec 5 How often the sender should drain the queue (in seconds)
logzioUrl https://listener.logz.io:8071 Logz.io URL, that can be found under "Log Shipping -> Libraries" in your account.
socketTimeout 10 * 1000 The socket timeout during log shipment
connectTimeout 10 * 1000 The connection timeout during log shipment
debug false Print some debug messages to stdout to help to diagnose issues
compressRequests false Boolean. true if logs are compressed in gzip format before sending. false if logs are sent uncompressed.

Parameters for in-memory queue

Parameter Default Explained
inMemoryQueueCapacityInBytes 1024 * 1024 * 100 The amount of memory(bytes) we are allowed to use for the memory queue. If the value is -1 the sender will not limit the queue size.
logsCountLimit -1 The number of logs in the memory queue before dropping new logs. Default value is -1 (the sender will not limit the queue by logs count)

Parameters for disk queue

Parameter Default Explained
queueDir None Where the sender should store the queue. It should be at least one folder in path.
fileSystemFullPercentThreshold 98 The percent of used file system space at which the sender will stop queueing. When we will reach that percentage, the file system in which the queue is stored will drop all new logs until the percentage of used space drops below that threshold. Set to -1 to never stop processing new logs
gcPersistedQueueFilesIntervalSeconds 30 How often the disk queue should clean sent logs from disk
checkDiskSpaceInterval 1000 How often the should disk queue check for space (in milliseconds)

Code Example

From version 1.0.15 we use a builder to get Logz.io sender

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.JsonObject;

public class LogzioSenderExample {

    public static void main(String[] args) {
        
        HttpsRequestConfiguration conf = HttpsRequestConfiguration
                        .builder()
                        .setLogzioListenerUrl("https://listener.logz.io:8071")
                        .setLogzioType("javaSenderType")
                        .setLogzioToken("123456789")
                        .build();
        
        // Use one of the following implementations
        // 1) disk queue example 
        LogzioSender logzioSender = LogzioSender
                        .builder()
                        .setTasksExecutor(Executors.newScheduledThreadPool(3))
                        .setReporter(new LogzioStatusReporter(){/*implement simple interface for logging sender logging */})
                        .setHttpsRequestConfiguration(httpsRequestConfiguration)
                        .withDiskQueue()
                            .setQueueDir(queueDir)
                        .endDiskQueue()
                        .build();
        
        // 2) in memory queue example
        LogzioSender logzioSender = LogzioSender
                        .builder()
                        .setTasksExecutor(Executors.newScheduledThreadPool(3))
                        .setDrainTimeoutSec(drainTimeout)
                        .setReporter(new LogzioStatusReporter(){/*implement simple interface for logging sender logging */})
                        .setHttpsRequestConfiguration(conf)
                        .withInMemoryQueue()
                        .endInMemoryQueue()
                        .build();

        sender.start();
        JsonObject jsonMessage = createLogMessage(); // create JsonObject to send to logz.io
        sender.send(jsonMessage);
    }
}

Until version 1.0.14

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.JsonObject;

public class LogzioSenderExample {

    public static void main(String[] args) {
        LogzioSender sender =  LogzioSender.getOrCreateSenderByType(token, logzioType, drainTimeoutSec,fileSystemFullPercentThreshold, queueDir,
                        logzioUrl, socketTimeout, serverTimeout, true, new LogzioStatusReporter(){/*implement simple interface for logging sender logging */}, Executors.newScheduledThreadPool(2),30);
        sender.start();
        JsonObject jsonMessage = createLogMessage(); // create JsonObject to send to logz.io
        sender.send(jsonMessage);
    }
}

Release notes

  • 1.1.0
    • remove deprecated getOrCreateSenderByType function
    • changed logzioSenderInstances Map key to hold an immutable pair of hashed token and log type
  • 1.0.19-20
    • minor dependency security updates
  • 1.0.18
    • add byte[] send option
  • 1.0.17
    • add support for log count limit option
  • 1.0.16
    • add implementation for in memory queue
  • 1.0.12
    • separating https request from the sender
    • add a builder for sender, http configuration, and queues implementation
  • 1.0.11 fix shaded
  • 1.0.10 add gzip compression
  • 1.0.9 add auto deploy
  • 1.0.8 fix st issue: Shaded library still listed as dependencies
  • 1.0.6 -1.0.7
    • add error message about reason of 400(BAD REQUEST)
  • 1.0.5
    • add runtime dependency on slf4j-api
    • fix NulPointerException
  • 1.0.1-1.0.4
    • Fix dependencies issue
  • 1.0.0
    • Initial release

Contribution

  • Fork
  • Code
  • mvn test
  • Issue a PR :)

About

Java library to send structured logs (JSONs) to Logz.io via HTTPS. This the core component of Logzio Appenders in the different logging frameworks (log4j, log4j2, Logback) which can also be used on its own

Topics

Resources

License

Packages

No packages published

Languages