/
architecture.html.md.erb
84 lines (50 loc) · 4.79 KB
/
architecture.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
---
title: Overview of the Loggregator System
owner: Logging and Metrics
---
<strong><%= modified_date %></strong>
Loggregator is the next generation system for aggregating and streaming logs and metrics from all of the user apps and system components in a<%= vars.product_full=='Cloud Foundry' ? "" : "n" %> <%= vars.product_full %> deployment.
View the [Loggregator repository on GitHub](https://github.com/cloudfoundry/loggregator).
# <a id='use-cases'></a> Using Loggregator
The main use cases are as follows:
- App developers can tail their application logs or dump the recent logs from the Cloud Foundry Command Line Interface (cf CLI), or stream these to a third party log archive and analysis service.
- Operators and administrators can access the **Loggregator [Firehose](#firehose)**, the combined stream of logs from all apps, and the metrics data from Cloud Foundry components.
- Operators can deploy "nozzles" to the Firehose. A nozzle is a component that listens to the Firehose for specified events and metrics, and streams this data to external services.
# <a id='use-cases'></a> Loggregator Components
<%= image_tag 'architecture/loggregator.png' %>
To see a larger version of this diagram, click <a href="./images/architecture/loggregator.png">here</a>.
<p class="note"><strong>Note</strong>: The Loggregator system now uses <a href="http://www.grpc.io/">gRPC</a> for communication between the Metron Agent and the Doppler, and between the Doppler and the Traffic Controller. This improves the stability and the performance of the Loggregator system, but it may require operators to scale their Dopplers.</p>
## <a id='sources'></a>Source
Sources are logging agents that run on the Cloud Foundry components.
## <a id='metron'></a>Metron
Metron agents are co-located with sources. They collect logs and forward them to the Doppler servers.
## <a id='doppler'></a>Doppler
Dopplers gather logs from the Metron agents, store them in temporary buffers, and forward them to the Traffic Controller or to third party syslog drains.
## <a id='traffic-controller'></a>Traffic Controller
Handles client requests for logs. Gathers and collates messages from all Doppler servers, and provides external API and message translation (as needed for legacy APIs). Exposes the Firehose.
## <a id='firehose'></a>Firehose
The Firehose is a WebSocket endpoint which streams all the event data coming from a<%= vars.product_full=='Cloud Foundry' ? "" : "n" %> <%= vars.product_full %> deployment. The data stream includes logs, HTTP events and container metrics from all applications, and metrics from all <%= vars.product_full %> system components. Logs from system components such as Cloud Controller are not included in the firehose and are typically accessed via rsyslog configuration.
Because the data coming from the Firehose may contain sensitive information, such as customer information in the application logs, the Firehose is only accessible by users who have the right permissions.
The Traffic Controller serves the Firehose over WebSocket at the `/firehose` endpoint. The events coming out of the Firehose are formatted as protobuf messages conforming to the [dropsonde protocol](https://github.com/cloudfoundry/dropsonde-protocol).
The address of the traffic controller can be discovered by hitting the `info` endpoint on the API and getting the value of the doppler\_logging\_endpoint.
Example output for a BOSH Lite CF environment:
<pre class='terminal'>
$ cf curl /v2/info | jq .doppler\_logging\_endpoint
wss://doppler.192.0.2.34.xip.io:443
</pre>
### <a id='logs-vs-metrics'></a>Difference Between Logs and Metrics
The Firehose carries both logs and metrics, which differ as follows:
* **Metrics**
- Report a measurement from or status of a VM at its timestamp time
- Follow [dropsonde](https://github.com/cloudfoundry/dropsonde-protocol) or [statsd](https://github.com/etsy/statsd) protocol
- Can route to a monitoring platform to trigger alerts
* **Logs**
- Report events detected, actions taken, errors, or any other messages the operator or developer wanted to generate
- Follow the syslog standard
- Are not used to trigger alerts
## <a id='nozzles'></a> Nozzles
Nozzles are programs which consume data from the Loggregator Firehose. Nozzles can be configured to select, buffer, and transform data, and forward it to other applications and services. For example:
- The JMX Bridge OpenTSDB Firehose Nozzle, which installs with [JMX Bridge](http://docs.pivotal.io/jmx-bridge)
- The [Datadog nozzle](https://github.com/cloudfoundry-incubator/datadog-firehose-nozzle) publishes metrics coming from the Firehose to Datadog.
- The [Syslog nozzle](https://github.com/cloudfoundry-community/firehose-to-syslog) filters out log messages coming from the Firehose and sends it to a syslog server.
See our [Nozzle Tutorial](./nozzle-tutorial.html).