Distributor

xinglang edited this page Feb 11, 2015 · 1 revision

Distributor is the stage to route events, it supports routing events to different destinations through Jetstream message, Kafka message. Distributor receives Pulsar raw events and session mark events from its previous stage Sessionizer. Distributor configures an Esper Processor, users can configure EPL to selectively filter/mutate/route events to different destinations.

Rules are configured in EPL and can be changed dynamically with configure app. Sample EPL looks like below:

@OutputTo("OutboundMessageChannel")
@ClusterAffinityTag(colname="si")
@PublishOn(topics="Pulsar.MC/ssnzEvent")	
select * from PulsarEvent;
	
@OutputTo("OutboundMessageChannel")
@PublishOn(topics="Pulsar.RR1/Mobile")
select * from PulsarEvent(et='Mobile');

It means route all PulsarEvent to Jetstream topic "Pulsar.MC/ssnzEvent" with affinity key "si" and route Mobile Pulsar events to Jetstream topic "Pulsar.RR1/Mobile". The subscriber clusters(Jetstream clusters) listen to those topic and get the events. We support both consistent hash based schedule with affinity key and round-robin based scheduler. The subscriber clusters need to create netty context and InboundMessage channels to receive the events.

Support Kafka Message

@OutputTo("PulsarKafkaChannel")
@PublishOn(topics="Pulsar.mobile")
select * from PulsarEvent(et='Mobile');

It means route Mobile events to kafka topic "Pulsar.mobile".

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.