Skip to content
Linux system data collector and send to target destination
Branch: master
Clone or download
Latest commit 70d92b0 Oct 26, 2016
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker Add Makefile and option 'readProcPath' for linkerConnector to read pr… Aug 25, 2016
sender [Implement]: Add dmidecode implement. Jul 22, 2016
test_data [Implement]: Add dmidecode implement. Jul 22, 2016
.gitignore Deployment script. Jul 22, 2016
.travis.yml fix go install. Oct 26, 2016
Makefile Add Makefile and option 'readProcPath' for linkerConnector to read pr… Aug 25, 2016
README.md Add Makefile and option 'readProcPath' for linkerConnector to read pr… Aug 25, 2016
collector.go Add Makefile and option 'readProcPath' for linkerConnector to read pr… Aug 25, 2016
collector_test.go fix go testing. Oct 26, 2016
data.go add stat back to machine and remove unnecessary Aug 23, 2016
debug.test [Implement]: Add dmidecode implement. Jul 22, 2016
dmidecode.go Fixed(Govet error) Aug 16, 2016
dmidecode_test.go [Fixed]: Skip test if dmidecode not exist. Jul 22, 2016
dockerBuild.sh Deployment script. Jul 22, 2016
main.go
makeLinux.sh Deployment script. Jul 22, 2016
runInDocker.sh [Implement]: Add dmidecode implement. Jul 22, 2016
testSendKafka.sh Deployment script. Jul 22, 2016

README.md

linkerConnector: Linux system data collector and forward to target destination.

Build Status

A tool to get Linux system data via /proc refer more detail on spec.

Install

Install if your system installed golang

go get -u -x github.com/LinkerNetworks/linkerConnector

Install using binary.

TBD

Usage

linkerConnector  (-server=XXX) (-dest=XXX) ...

## Check all command destription
linkerConnector --help

## optional: using make
make compile-static-amd64: static compile
make docker-build: build a docker image whose name is linkerrepository/linker_connector
make container-run: run the image built above
make container-stop: stop container
make container-clean: remove container
make clean: clean binary file

Options

  • readProcPath/r : File path of proc for linkerConnector to read from.
  • cAdvisorAddr/c : Http Url for cAdvisor REST API.
  • server/s : The comma separated list of server could be brokers in the Kafka cluster or spark address.
  • topic/t : The topic to kafka produce.
  • interval/i : Time interval(second) to retrieval data , default 0 is not repeat.
  • dest/d : Destination to kafka, spark, cassandra and stdout.
  • pipe/p: Using pipe mode to forward data. usage cat data.txt | linkerConnector -p true
  • dsiableFileSave / d: Disable local file save.
  • --help : Display detail command help.

Important Data

  • Disk I/O /proc/{PID}/io
  • Process Status: (Zombie, VM Data) /proc/{PID}/status.
    • state (R is running, S is sleeping, D is sleeping in an uninterruptible wait, Z is zombie, T is traced or stopped).
    • Zombie: No parent process.
  • How to find parent process:
    • ps axjf (Linux only)
  • Hardware Infomation Dump:
    • dmidecode, only dump memory dmidecode -- type memory

TODOs

  • DataCollect
    • ProcessInfo
      • Proc Information
      • File Info
      • Host's all Container Info
    • MachineIndo
      • CPUInfo
      • MemInfo
      • DMIINFO- Bios Information
      • DMIINFO- System Information
      • NetInfo
      • DiskInfo
    • Last Login history
    • Syslog
  • Data Forwarding
    • Kafka
    • Spark Streaming
    • Stdout
    • Pipe mode
    • Cassandra
    • TBC
  • Data Generator
  • Randomize data generator
You can’t perform that action at this time.