Range is persistent storage of streams of records. It is a horizontally-scalable, highly-available, blazinbly fast aggregation system of streams of records.
Range is designed to store unlimited number of sequentual records from thousands of sources. Due to its simplicity, range is very cost effective solution what makes it to be an effective tool for storing logging data, dozens of millions records per second. Range is used by logrange - distributed and extremely fast log aggregation system.
Range's highlights:
- Aggregation system of streams of records
- Sstand-alone and Distributed configurations are available
- Streams size doesn't impact system performance
- Range is able to work with hundreds of thousands of streams
- Created for and support Security of various systems
- Optimized for storing dozens of millions records per second
Range could be used as a library. In this configuration the streams are persisted on local file-system. Records are available via journals - persisted streams of records. The following piece of code illustrates, how it works:
package main
import (
"log"
"context"
"os"
"github.com/logrange/range/embed"
"github.com/logrange/range/pkg/records"
)
func main() {
// Creating a new journal controller in os.Arg[1] directory
jc, err := embed.NewJCtrlr(embed.JCtrlrConfig{JournalsDir: os.Arg[1]})
if err != nil {
log.Fatal("Could not create new controller, err=", err)
}
defer jc.Close()
// Obtaining the journal by its name "teststream"
j, err := jc.GetOrCreate(context.Background(), "teststream")
if err != nil {
log.Fatal("could not get new journal err=", err)
}
// Writes couple records "aaaa" and "bbbb" into the journal
_, _, err = j.Write(context.Background(), records.SrtingsIterator("aaaa", "bbbb"))
if err != nil {
log.Fatal("Write err=", err)
}
// telling the journal to call fsync(), to be able to iterate over the fresh data right now
j.Sync()
// obtaining iterator to read the journal
it := j.Iterator()
r, _ := it.Get(context.Background())
log.Println("First record ", string(r))
it.Next(context.Background())
r, _ = it.Get(context.Background())
log.Println("Second record ", string(r))
// jump to the first record again
pos := it.Pos()
pos.Idx = 0
it.SetPos(pos)
r, _ = it.Get(context.Background())
log.Println("First recor ", string(r))
// release resources
it.Close()
}
This project is licensed under the Apache Version 2.0 License - see the LICENSE file for details
- GoLand IDE by JetBrains is used for the code development