Linux system data collector and send to target destination
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker
sender
test_data
.gitignore
.travis.yml
Makefile
README.md
collector.go
collector_test.go
data.go
debug.test
dmidecode.go
dmidecode_test.go
dockerBuild.sh
main.go
makeLinux.sh
runInDocker.sh
testSendKafka.sh

README.md

linkerConnector: Linux system data collector and forward to target destination.

Build Status

A tool to get Linux system data via /proc refer more detail on spec.

Install

Install if your system installed golang

go get -u -x github.com/LinkerNetworks/linkerConnector

Install using binary.

TBD

Usage

linkerConnector  (-server=XXX) (-dest=XXX) ...

## Check all command destription
linkerConnector --help

## optional: using make
make compile-static-amd64: static compile
make docker-build: build a docker image whose name is linkerrepository/linker_connector
make container-run: run the image built above
make container-stop: stop container
make container-clean: remove container
make clean: clean binary file

Options

  • readProcPath/r : File path of proc for linkerConnector to read from.
  • cAdvisorAddr/c : Http Url for cAdvisor REST API.
  • server/s : The comma separated list of server could be brokers in the Kafka cluster or spark address.
  • topic/t : The topic to kafka produce.
  • interval/i : Time interval(second) to retrieval data , default 0 is not repeat.
  • dest/d : Destination to kafka, spark, cassandra and stdout.
  • pipe/p: Using pipe mode to forward data. usage cat data.txt | linkerConnector -p true
  • dsiableFileSave / d: Disable local file save.
  • --help : Display detail command help.

Important Data

  • Disk I/O /proc/{PID}/io
  • Process Status: (Zombie, VM Data) /proc/{PID}/status.
    • state (R is running, S is sleeping, D is sleeping in an uninterruptible wait, Z is zombie, T is traced or stopped).
    • Zombie: No parent process.
  • How to find parent process:
    • ps axjf (Linux only)
  • Hardware Infomation Dump:
    • dmidecode, only dump memory dmidecode -- type memory

TODOs

  • DataCollect
    • ProcessInfo
      • Proc Information
      • File Info
      • Host's all Container Info
    • MachineIndo
      • CPUInfo
      • MemInfo
      • DMIINFO- Bios Information
      • DMIINFO- System Information
      • NetInfo
      • DiskInfo
    • Last Login history
    • Syslog
  • Data Forwarding
    • Kafka
    • Spark Streaming
    • Stdout
    • Pipe mode
    • Cassandra
    • TBC
  • Data Generator
  • Randomize data generator