avroipc

Avroipc is a pure-go-implemented client for flume's avro source.

I wrote avroipc to learn avro rpc protocol, and it's not production ready!, use it as you own risk.

Thanks to Linkedin's goavro!

Usage

package main

import (
    "log"

    "github.com/myzhan/avroipc"
)

func main() {
    // Create a new client with default parameters
    client, err := avroipc.NewClient("localhost:20200")
    if err != nil {
        log.Fatal(err)
    }
    
    event := &avroipc.Event{
        Body: []byte("hello from go"),
    	Headers: map[string]string {
            "topic": "myzhan",
            "timestamp": "1508740315478",
        },
    }
    status, err := client.Append(event)
    if err != nil {
        log.Fatal(err)
    }
    if status != "OK" {
        log.Fatalf("Bad status: %s", status)
    }
}

To specify particular parameters of the client it is possible to use the config builder:

package main
import (
    "log"
    "time"

    "github.com/myzhan/avroipc"
)

func main() {
    config := avroipc.NewConfig()
    config.WithTimeout(3*time.Second)

    client, err := avroipc.NewClientWithConfig("localhost:20200", config)
    if err != nil {
        log.Fatal(err)
    }
    
    // Use the client as before
    _ = client
}

Development

Clone the repository and do the following sequence of command:

go get
go test ./...

To run a test with a real client run the following command:

FLUME_SERVER_ADDRESS=127.0.0.1:20201 go test -count=1 -run TestSend

where 127.0.0.1:20201 is a real Apache Flume server, -count=1 is a way to disable Go build cache.

If you want to run a test with a real client and enabled data compression run the following command:

FLUME_SERVER_ADDRESS=127.0.0.1:20201 FLUME_COMPRESSION_LEVEL=1 go test -count=1 -run TestSend

where FLUME_COMPRESSION_LEVEL is a new environment variable to specify wanted compression level. Support values from 1 to 9.

License

Open source licensed under the MIT license (see LICENSE file for details).