Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
539 lines (516 sloc) 17 KB
---
title: Configuring System Logging
owner: Logging and Metrics
---
<strong><%= modified_date %></strong>
This topic explains how to configure the Cloud Foundry [Loggregator system](../../loggregator/architecture.html).
## <a id='scaling'></a> Scaling Loggregator ##
Cloud Foundry system components and apps constantly generate log and metrics data. The [Metron](../../loggregator/architecture.html#metron) agent running on each component or application VM collects and sends this data out to [Doppler](../../loggregator/architecture.html#doppler) components, which temporarily buffer the data before periodically forwarding it to the [Traffic Controller](../../loggregator/architecture.html#traffic-controller). The Traffic Controller then serves the aggregated data stream through the Firehose WebSocket endpoint.
When the log and metrics data input to a Doppler exceeds its buffer size for a given interval, data can be lost. You can take several actions to minimize this loss.
### Add additional Doppler server instances
You can increase the number of Doppler servers by increasing the `instances` property for the `doppler_z1` and `doppler_z2` jobs in your Cloud Foundry BOSH deployment manifest.
## <a id='syslog-forward'></a> Enabling System Log Forwarding ##
Cloud Foundry can forward syslog data to an external aggregator using the [syslog-release](https://github.com/cloudfoundry/syslog-release).
## <a id='customizing'></a>Customizing Loggregator Components ##
You can customize each Loggregator component by changing its properties in the CF deployment manifest. The following details some of the most commonly used changes.
### Doppler ###
<table>
<tr>
<th><strong>Property</strong></th>
<th><strong>Description</strong></th>
<th><strong>Default</strong></th>
</tr>
<tr>
<td>doppler.zone</td>
<td>Zone of the doppler server</td>
<td>no default</td>
</tr>
<tr>
<td>doppler.debug</td>
<td>Boolean value to enable verbose logging for Diego and Doppler server (the Doppler system)</td>
<td><code>false</code></td>
</tr>
<tr>
<td>doppler.maxRetainedLogMessages</td>
<td>Number of log messages to retain per application</td>
<td><code>100</code></td>
</tr>
<tr>
<td>doppler.dropsonde_incoming_port</td>
<td>Port for incoming messages in the dropsonde format</td>
<td><code>3457</code></td>
</tr>
<tr>
<td>doppler.incoming_tcp_port</td>
<td>Port for incoming TCP messages</td>
<td><code>3458</code></td>
</tr>
<tr>
<td>doppler.tls.enable</td>
<td>Enable TLS listener on doppler so it can receive dropsonde envelopes over TLS transport. If enabled, you must specify Certificate and Key files.</td>
<td><code>false</code></td>
</tr>
<tr>
<td>doppler.tls.port</td>
<td>Port for incoming messages in the dropsonde format over TLS listener</td>
<td><code>3459</code></td>
</tr>
<tr>
<td>doppler.tls.server_cert</td>
<td>TLS server certificate</td>
<td>no default</td>
</tr>
<tr>
<td>doppler.tls.server_key</td>
<td>TLS server key</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.tls.ca_cert</td>
<td>CA root required for key/certificate verification</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.etcd.require_ssl</td>
<td>Enable SSL for all communication with ETCD</td>
<td><code>false</code></td>
</tr>
<tr>
<td>loggregator.etcd.ca_cert</td>
<td>PEM-encoded CA certificate</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.etcd.client_cert</td>
<td>PEM-encoded client certificate</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.etcd.client_key</td>
<td>PEM-encoded client key</td>
<td>no default</td>
</tr>
<tr>
<td>doppler.outgoing_port</td>
<td>Port for outgoing log messages</td>
<td><code>8081</code></td>
</tr>
<tr>
<td>doppler.websocket_write_timeout_seconds</td>
<td>Interval before aborting unsuccessful WebSocket write</td>
<td><code>60</code></td>
</tr>
<tr>
<td>doppler.blacklisted_syslog_ranges</td>
<td>Blacklist for IP addresses that should not be used as syslog drains. For example, internal IP addresses</td>
<td>no default</td>
</tr>
<tr>
<td>doppler.container_metric_ttl_seconds</td>
<td>Time to live (TTL), in seconds, for container usage metrics</td>
<td><code>120</code></td>
</tr>
<tr>
<td>doppler.unmarshaller_count</td>
<td>Number of parallel unmarshallers to run within Doppler</td>
<td><code>5</code></td>
</tr>
<tr>
<td>doppler.sink_inactivity_timeout_seconds</td>
<td>Interval before removing a sink due to inactivity</td>
<td><code>3600</code></td>
</tr>
<tr>
<td>doppler.sink_dial_timeout_seconds</td>
<td>Dial timeout for sinks</td>
<td><code>1</code></td>
</tr>
<tr>
<td>doppler.sink_io_timeout_seconds</td>
<td>I/O Timeout on sinks</td>
<td><code>0</code></td>
</tr>
<tr>
<td>doppler_endpoint.shared_secret</td>
<td>Shared secret used to verify cryptographically signed dropsonde messages</td>
<td>no default</td>
</tr>
<tr>
<td>doppler.message_drain_buffer_size</td>
<td>Size of the internal buffer used by doppler to store messages for output to firehose or Cloud Foundry logs. If buffer fills, Doppler drops messages.</td>
<td><code>10000</code></td>
</tr>
<tr>
<td>doppler.syslog_skip_cert_verify</td>
<td>Boolean value to disable certificate verification for syslog sink when connecting over TLS</td>
<td><code>true</code></td>
</tr>
<tr>
<td>doppler.locked_memory_limit</td>
<td>Shell's locked memory limit size. Accepts numeric values interpreted as KsB, or the following non-numeric values: <code>kernel</code>, <code>soft</code>, <code>hard</code>, <code>unlimited</code>. <code>kernel</code> sets limit to the kernel's default.</td>
<td><code>unlimited</code></td>
</tr>
<tr>
<td>loggregator.etcd.machines</td>
<td>IP addresses pointing to the ETCD cluster</td>
<td>no default</td>
</tr>
<tr>
<td>metron_endpoint.host</td>
<td>Host used to emit messages to the Metron agent</td>
<td><code>127.0.0.1</code></td>
</tr>
<tr>
<td>metron_endpoint.dropsonde_port</td>
<td>Port used to emit dropsonde messages to the Metron agent</td>
<td><code>3457</code></td>
</tr>
</table>
### Traffic Controller ###
<table>
<tr>
<th><strong>Property Name</strong></th>
<th><strong>Description</strong></th>
<th><strong>Default</strong></th>
<tr>
<td>traffic_controller.debug</td>
<td>Boolean value to enable verbose logging for Diego and Loggregator server (the Loggregator system)</td>
<td><code>false</code></td>
</tr>
<tr>
<td>traffic_controller.disable_access_control</td>
<td>Boolean value to set Traffic Controller to bypasses authentication with the UAA and Cloud Controller</td>
<td><code>false</code></td>
</tr>
<tr>
<td>traffic_controller.locked_memory_limit</td>
<td>Shell's locked memory limit size. Accepts numeric values interpreted as KsB, or the following non-numeric values: <code>kernel</code>, <code>soft</code>, <code>hard</code>, <code>unlimited</code>. <code>kernel</code> sets limit to the kernel's default.</td>
<td><code>unlimited</code></td>
</tr>
<tr>
<td>loggregator.outgoing_dropsonde_port</td>
<td>Port for outgoing dropsonde messages</td>
<td><code>8081</code></td>
</tr>
<tr>
<td>traffic_controller.security_event_logging.enabled</td>
<td>Boolean value to enable logging of all requests made to the Traffic Controller in CEF format</td>
<td><code>false</code></td>
</tr>
<tr>
<td>doppler.uaa_client_id</td>
<td>Doppler's client id to connect to UAA</td>
<td><code>doppler</code></td>
</tr>
<tr>
<td>uaa.clients.doppler.secret</td>
<td>Doppler's client secret to connect to UAA</td>
<td>no default</td>
</tr>
<tr>
<td>uaa.url</td>
<td>URL of UAA</td>
<td>no default</td>
</tr>
<tr>
<td>login.protocol</td>
<td>Protocol to use to connect to UAA if uaa.url is not set</td>
<td><code>https</code></td>
</tr>
<tr>
<td>loggregator.etcd.require_ssl</td>
<td>Enable SSL for all communication with ETCD</td>
<td><code>false</code></td>
</tr>
<tr>
<td>loggregator.etcd.machines</td>
<td>IP addresses pointing to the ETCD cluster</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.etcd.maxconcurrentrequests</td>
<td>Maximum number of concurrent requests to ETCD</td>
<td><code>10</code></td>
</tr>
<tr>
<td>loggregator.etcd.ca_cert</td>
<td>PEM-encoded CA certificate</td>
<td>no default</td>
</tr>
<tr>
<td>traffic_controller.etcd.client_cert</td>
<td>PEM-encoded client certificate</td>
<td>no default</td>
</tr>
<tr>
<td>traffic_controller.etcd.client_key</td>
<td>PEM-encoded client key</td>
<td>no default</td>
</tr>
<tr>
<td>traffic_controller.pprof_port</td>
<td>pprof port for runtime profiling data</td>
<td><code>6060</code></td>
</tr>
<tr>
<td>system_domain</td>
<td>Domain reserved for Cloud Foundry operator, and the base URL where login, UAA, and other non-user apps listen</td>
<td>no default</td>
</tr>
<tr>
<td>ssl.skip_cert_verify</td>
<td>Boolean value to ignore bad SSL certificates when connecting over https</td>
<td><code>false</code></td>
</tr>
<tr>
<td>cc.srv_api_uri</td>
<td>API URI of Cloud Controller</td>
<td>no default</td>
</tr>
</table>
### Metron Agent ###
<table>
<tr>
<th><strong>Property Name</strong></th>
<th><strong>Description</strong></th>
<th><strong>Default</strong></th>
</tr>
<tr>
<td>syslog_daemon_config.enable</td>
<td>Boolean value to enable rsyslog configuration for forwarding syslog messages into Metron</td>
<td><code>true</code></td>
</tr>
<tr>
<td>syslog_daemon_config.address</td>
<td>IP address for syslog aggregator</td>
<td>no default</td>
</tr>
<tr>
<td>syslog_daemon_config.port</td>
<td>TCP port of syslog aggregator</td>
<td>no default</td>
</tr>
<tr>
<td>syslog_daemon_config.transport</td>
<td>Transport to use when forwarding logs. Accepts the following values: <code>tcp</code>, <code>udp</code>, or <code>relp</code></td>
<td><code>tcp</code></td>
</tr>
<tr>
<td>syslog_daemon_config.fallback_addresses</td>
<td>IP addresses of fallback servers to use if primary syslog server is unavailable. Only </code>tcp</code> and <code>relp</code> supported. Each list entry must consist of address, transport, and port keys.</td>
<td><code>[]</code></td>
</tr>
<tr>
<td>syslog_daemon_config.custom_rule</td>
<td>Custom rule for syslog forward daemon</td>
<td>no default</td>
</tr>
<tr>
<td>syslog_daemon_config.max_message_size</td>
<td>Maximum message size</td>
<td><code>4k</code></td>
</tr>
<tr>
<td>metron_endpoint.shared_secret</td>
<td>Shared secret used to verify cryptographically signed dropsonde messages</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.listening_port</td>
<td>Port on which the Metron agent listens to receive dropsonde log messages</td>
<td><code>3457</code></td>
</tr>
<tr>
<td>metron_agent.listening_address</td>
<td>IP address on which the Metron agent listens to receive dropsonde log messages provided for BOSH links, should not be overwritten</td>
<td><code>127.0.0.1</code></td>
</tr>
<tr>
<td>metron_agent.debug</td>
<td>Boolean value to enable verbose mode</td>
<td><code>false</code></td>
</tr>
<tr>
<td>metron_agent.protocols</td>
<td>A priority list of protocols over which Metron connects to Doppler. Metron will refuse to connect to Doppler over any protocol not on this list.</td>
<td><code>["udp"]</code></td>
</tr>
<tr>
<td>metron_agent.tls.client_cert</td>
<td>TLS client certificate</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.tls.client_key</td>
<td>TLS client key</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.tls.ca_cert</td>
<td>CA root required for key/certificate verification</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.zone</td>
<td>Availability zone where this agent runs</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.deployment</td>
<td>Name of deployment. Added as tag on all outgoing metrics.
<td>no default</td>
</tr>
<tr>
<td>metron_agent.tcp.batching_buffer_bytes</td>
<td>Number of bytes which can be buffered prior to TCP write. This applies to TLS over TCP.</td>
<td><code>10240</code></td>
</tr>
<tr>
<td>metron_agent.tcp.batching_buffer_flush_interval_milliseconds</td>
<td>Maximum time a message can stay in the batching buffer before being flushed</td>
<td><code>100</code></td>
</tr>
<tr>
<td>metron_agent.logrotate.freq_min</td>
<td>Frequency, in minutes, with which logrotate rotates VM logs</td>
<td><code>5</code></td>
</tr>
<tr>
<td>metron_agent.logrotate.rotate</td>
<td>Number of files that logrotate retains on the VM</td>
<td><code>7</code></td>
</tr>
<tr>
<td>metron_agent.logrotate.size</td>
<td>Size at which logrotate rotates log file</td>
<td><code>50M</code></td>
</tr>
<tr>
<td>loggregator.etcd.require_ssl</td>
<td>Boolean value to enable SSL for all communication with ETCD</td>
<td><code>false</code></td>
</tr>
<tr>
<td>loggregator.etcd.machines</td>
<td>IP addresses pointing to the ETCD cluster</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.etcd.maxconcurrentrequests</td>
<td>Maximum number of concurrent requests to ETCD</td>
<td><code>10</code></td>
</tr>
<tr>
<td>loggregator.etcd.ca_cert</td>
<td>PEM-encoded CA certificate</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.etcd.client_cert</td>
<td>PEM-encoded client certificate</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.etcd.client_key</td>
<td>PEM-encoded client key</td>
<td>no default</td>
</tr>
<tr>
<td>metron_agent.pprof_port</td>
<td>pprof port for runtime profiling data</td>
<td><code>6061</code></td>
</tr>
</table>
### Syslog Drain Binder###
See [Using Log Management Services](../../devguide/services/log-management.html).
<table>
<tr>
<th><strong>Property Name</strong></th>
<th><strong>Description</strong></th>
<th><strong>Default</strong></th>
</tr>
<td>metron_endpoint.host</td>
<td>Host used to emit messages to the Metron agent</td>
<td><code>127.0.0.1</code></td>
</tr>
<tr>
<td>metron_endpoint.dropsonde_port</td>
<td>Port used to emit dropsonde messages to the Metron agent</td>
<td><code>3457</code></td>
</tr>
<tr>
<td>loggregator.etcd.require_ssl</td>
<td>Boolean value to enable SSL for all communication with ETCD</td>
<td><code>false</code></td>
</tr>
<tr>
<td>loggregator.etcd.machines</td>
<td>IP addresses pointing to the ETCD cluster</td>
<td>no default</td>
</tr>
<tr>
<td>loggregator.etcd.maxconcurrentrequests</td>
<td>Maximum number of concurrent requests to ETCD</td>
<td><code>10</code></td>
</tr>
<tr>
<td>loggregator.etcd.ca_cert</td>
<td>PEM-encoded CA certificate</td>
<td>no default</td>
</tr>
<tr>
<td>syslog_drain_binder.etcd.client_cert</td>
<td>PEM-encoded client certificate</td>
<td>no default</td>
</tr>
<tr>
<td>syslog_drain_binder.etcd.client_key</td>
<td>PEM-encoded client key</td>
<td>no default</td>
</tr>
<tr>
<td>system_domain</td>
<td>Domain reserved for Cloud Foundry operator, and the base URL where login, UAA, and other non-user apps listen </td>
<td>no default</td>
</tr>
<tr>
<td>syslog_drain_binder.drain_url_ttl_seconds</td>
<td>Time to live (TTL), in seconds, for drain URLs</td>
<td><code>60</code></td>
</tr>
<tr>
<td>syslog_drain_binder.update_interval_seconds</td>
<td>Interval, in seconds, on which to poll Cloud Controller</td>
<td><code>15</code></td>
</tr>
<tr>
<td>syslog_drain_binder.polling_batch_size</td>
<td>Batch size for the poll from Cloud Controller</td>
<td><code>1000</code></td>
</tr>
<tr>
<td>syslog_drain_binder.debug</td>
<td>Boolean value to enable verbose logging for syslog_drain_binder</td>
<td><code>false</code></td>
</tr>
<tr>
<td>syslog_drain_binder.locked_memory_limit</td>
<td>Shell's locked memory limit size. Accepts numeric values interpreted as KsB, or the following non-numeric values: <code>kernel</code>, <code>soft</code>, <code>hard</code>, <code>unlimited</code>. <code>kernel</code> sets limit to the kernel's default.</td>
<td><code>unlimited</code></td>
</tr>
<tr>
<td>cc.bulk_api_password</td>
<td>Password for the bulk API</td>
<td>no default</td>
</tr>
<tr>
<td>cc.srv_api_uri</td>
<td>API URI of Cloud Controller</td>
<td>no default</td>
</tr>
<tr>
<td>ssl.skip_cert_verify</td>
<td>Boolean value to ignore bad SSL certificates when connecting over https</td>
<td><code>false</code></td>
</tr>
</table>