Skip to content

nickman/HeliosStreams

Repository files navigation

HeliosStreams

Streaming performance data all the way home.

HeliosStreams Architecture

Data Types

Data Format Specifications

Value Types

Symbol Name Type Description Parameters
A Accumulator Valueless Tracking an all time total of a specific event type. None
M Meter Valueless Tracking the per seconds/minutes/hours/day rate of a specific event type None
D Delta Value The value of a given metric instance is the delta of the current and the prior value TBD
P Period Aggregated Value Values are aggregated to count, min, max and average per defined period (e.g. 15 sedonds) TBD
S Straight Through Value The metric is not aggregated and forwarded directly to endpoint None

Plain Text

The plain text stream is typically used when extracting data from application log or some other unstructured data feed after performing an extract-and-format operation in an intermediary such as LogStash and/or Filebeat.

Data Type Routing

  • Directed: The line of text is beig submitted to a stream that is specific to the data type of the metric.
  • Undirected: The line of text contains a symbol representing the data type of the metric and will be routed to an appropriate stream processor according to that symbol.

Text Line Format

[<value-type>,]<timestamp>, [<value>,] <metric-name>, <host>, <app> [,<tagkey1>=<tagvalue1>,<tagkeyn*>=<tagvaluen>]*

  • value-type: The value type symbol indicating how the metric should be routed. Only required if the routing is undirected.
  • timestamp: The effective timestamp of the metric in seconds-since-the epoch or milliseconds-since-the epoch.
  • value: The metric numeric value. Absent in valueless types.
  • metric-name: The identifier of the meaning of the metric. The fully qualified (global) name is comprised of the metric-name, the host, the app and the tags.
  • host : The name of the host the metric originated from
  • app : The name of the application the metric originated from
  • tags : Key value pairs that qualify the instance of the metric-name.