Skip to content

Architecture

Jump to bottom Edit New page
elenalazovik edited this page Dec 22, 2014 · 34 revisions

#Concept of Particles

Apache Storm works with tuples every one of which is a data item to be processed. The Particles (that is how we name our primitives) represent a special type of tuple. The features of Particle are:

  • Every Particle has a timestamp;
  • Every Particle is strongly typed.

The Particles are matched to Apache Storm tuples automatically, therefore, there is no any break of compatibility with the basic Storm functionality.

We distinguish 2(two) different types of Particles: Data Particle and Meta Particle (see Figure 1). They both are processed by Storm cluster, but in a different way. Figure 1. Data and Meta Particles ##Data Particles raw data streams

##Meta Particles broadcast of particles of this type

###Grouper for MetaParticles

##Sync buffer

  • apply ordering -use time window (now const) -duplicate Metaparticles filter if too late: rejected
Add a custom footer

Clone this wiki locally