Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

a flume agent,thrift+c#,multi collector supported

branch: master

pretty readme

latest commit f3b3a3fd0d
Medcl authored March 05, 2012
Octocat-spinner-32 FlumeAgent initial commit March 05, 2012
Octocat-spinner-32 UnitTest initial commit March 05, 2012
Octocat-spinner-32 .gitignore initial commit March 05, 2012
Octocat-spinner-32 README.textile pretty readme March 05, 2012
README.textile

Flume .NET Agent (thrift client)

License

Apache2

Requirements

tested in flume 0.9

Tutorial

config the flume’s source and sink ,
open your flume master’s config page,for example:
http://localhost:35871/flumeconfig.jsp [flume master]


collector_name1 : thriftSource( 2014 )| collectorSink(“hdfs://localhost/flume/%{catalog}/2012”, "",5000,raw());
collector_name2 : thriftSource( 2015 )| collectorSink(“hdfs://localhost/flume/%{catalog}/2012”, "",5000,raw());

and then map the collectors to the physical/logical node,
http://localhost:35871/mastercommand.jsp

map your_physical_node_name1 collector_name1
map your_physical_node_name2 collector_name2


ps: suppose the ip and node name is mapped as below:
your_physical_node_name1 == 10.1.1.125
your_physical_node_name2 == 10.1.1.126

and then,you can send your logs to the flume server through the agent now.


//init config
FlumeConfig.Instance.AddCollector(new Collector(){Name = “collector1”, ThriftNodes = new FlumeNodeConfig[]{new FlumeNodeConfig(){Host = “10.1.1.125”,Port = 2014,Enabled = true}, }});
FlumeConfig.Instance.AddCollector(new Collector(){Name = “collector2”, ThriftNodes = new FlumeNodeConfig[]{new FlumeNodeConfig(){Host = “10.1.1.126”,Port = 2015,Enabled = true}, }});
FlumeConfig.Instance.AddSources(new Source(){Name = “source1”,Collector = "collector1"});
FlumeConfig.Instance.AddSources(new Source(){Name = “source2”,Collector = "collector2"});

//collecting
FlumeAgent. FlumeAgent agent=new FlumeAgent.FlumeAgent();
agent.Send(“source1”, “hello,world,datata,blala…..”);
agent.Send(“source2”, “test from agent…..”);

Have fun!

Something went wrong with that request. Please try again.