klogstream

klogstream is a Go library that implements Kubernetes log streaming with multi-pod filtering, JSON formatting, and error handling. The library extends standard Kubernetes clients by adding regex-based filtering and concurrent log processing. It provides a handler interface for custom log processing, automatic reconnection mechanisms, and built-in support for multiline log assembly. Developers can filter logs by pod names, namespaces, and labels while streaming from multiple containers simultaneously.

Features

Concurrent log streaming across multiple pods/containers using goroutines
Regex-based filtering for pod/container names
Namespace and label-based log filtering
Multiline log reassembly (e.g., Java stack traces)
Flexible log formatting with JSON and custom formats
Pluggable log handler system
Automatic reconnection with exponential backoff
Direct Kubernetes clientset injection support for testing

Installation

go get github.com/archsyscall/klogstream/pkg/klogstream

Log Handler Interface

klogstream provides a LogHandler interface for flexible log processing:

type LogHandler interface {
    // OnLog is called whenever a new log message arrives
    OnLog(message LogMessage)
    
    // OnError is called when an error occurs during streaming
    OnError(err error)
    
    // OnEnd is called when the streaming ends
    OnEnd()
}

LogMessage structure:

type LogMessage struct {
    Timestamp     time.Time
    PodName       string
    ContainerName string
    Message       string
    Labels        map[string]string
}

Quick Start

package main

import (
	"context"
	"fmt"
	"time"
	// Other imports...

	"github.com/archsyscall/klogstream/pkg/klogstream"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()
	
	// Set up context and signal handling...

	// Create a streamer using the builder pattern - this is the main part!
	streamer, err := klogstream.NewBuilder().
		WithNamespace("default").             // Stream logs from the default namespace
		WithPodRegex("my-app.*").             // Stream logs from pods matching regex
		WithContainerRegex(".*").             // Stream logs from all containers
		WithHandler(&ConsoleHandler{}).       // Use a custom log handler
		Build()

	if err != nil {
		// Error handling...
	}

	// Start streaming logs
	if err := streamer.Start(ctx); err != nil && err != context.Canceled {
		// Error handling...
	}
}

// ConsoleHandler is a simple handler that prints logs to the console
type ConsoleHandler struct{}

func (h *ConsoleHandler) OnLog(message klogstream.LogMessage) {
	fmt.Printf("[%s] %s/%s: %s\n", 
		message.Timestamp.Format(time.RFC3339),
		message.PodName,
		message.ContainerName,
		message.Message)
}

// OnError and OnEnd implementations...

Development Status

This project is under active development and not yet production-ready. APIs may change without notice.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT

License

archsyscall/klogstream

ErrorLooks like something went wrong!

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Languages

Error
Looks like something went wrong!

Packages