Fetching contributors…
Cannot retrieve contributors at this time
124 lines (77 sloc) 7.08 KB
title: Streaming Application Logs to Log Management Services
owner: PCF Metrics
<strong><%= modified_date %></strong>
This topic describes how to drain logs from Cloud Foundry to a third-party log management service.
Cloud Foundry aggregates logs for all instances of your applications as well as for requests made to your applications through internal components of Cloud Foundry. For example, when the Cloud Foundry Router forwards a request to an application, the Router records that event in the log stream for that app. Run the following command to access the log stream for an app in the terminal:
<pre class='terminal'>$ cf logs YOUR-APP-NAME</pre>
If you want to persist more than the limited amount of logging information that Cloud Foundry can buffer, drain these logs to a log management service.
For more information about the systems responsible for log aggregation and streaming in Cloud Foundry, see [Application Logging in Cloud Foundry](../deploy-apps/streaming-logs.html).
## <a id='marketplace-services'></a>Using Services from the Cloud Foundry Marketplace
Your Cloud Foundry marketplace may offer one or more log management services. To use one of these services, create an instance of the service and bind it to your application with the following commands:
<pre class='terminal'>
$ cf bind-service YOUR-APP YOUR-LOG-STORE
For more information about service instance lifecycle management, see the [Managing Service Instances](managing-services.html) topic.
<p class='note'><strong>Note</strong>: Not all marketplace services support syslog drains. Some services implement an integration with Cloud Foundry that enables automated streaming of application syslogs. <%= vars.custom_services %></p>
## <a id='user-provided'></a>Using Services Not Available in Your Marketplace
If a compatible log management service is not available in your Cloud Foundry marketplace, you can use [user-provided service instances](./user-provided.html#syslog) to stream application logs to a service of your choice.
You can install and use the [CF Drain CLI Plugin](#drain-plugin) to create and manage user-provided syslog drains from the CF command-line interface (cf CLI).
You may need to prepare your log management service to receive app logs from Cloud Foundry. For specific instructions for several popular services, see [Service-Specific Instructions for Streaming Application Logs](./log-management-thirdparty-svc.html). If you cannot find instructions for your service, follow the generic instructions below.
### <a id='step1'></a>Step 1: Configure the Log Management Service
Complete the following steps to set up a communication channel between the log management service and your Cloud Foundry deployment:
1. Obtain the <%=vars.external_ips%> that <%=vars.admin%> assigns to outbound traffic.
1. Provide these IP addresses to the log management service.
The specific steps to configure a third-party log management service depend on
the service.
1. Whitelist these IP addresses to ensure unrestricted log routing to your log management service.
1. Record the syslog URL provided by the third-party service. Third-party services
typically provide a syslog URL to use as an endpoint for incoming log data. You use this syslog URL in Step 2: Create a User-provided Service Instance.
Cloud Foundry uses the syslog URL to route messages to the service.
The syslog URL has a scheme of `syslog`, `syslog-tls`, or `https`, and can
include a port number.
For example:
### <a id='step2'></a>Step 2: Create and Bind a User-Provided Service Instance
You can create a syslog drain service and bind apps to it using either generic Cloud Foundry Command Line Interface (cf CLI) commands, or drain-specific commands enabled by the CF Drain plugin for the cf CLI.
Each option is described below.
#### <a id='drain-plugin'></a>With the CF Drain CLI Plugin
1. If the CF Drain CLI Plugin is not installed on your local workstation, follow the [Installing Plugin]( instructions in the plugin source repository.
2. Run `cf create-drain APP-NAME DRAIN-NAME SYSLOG-DRAIN-URL`, filling in values as follows:
- APP-NAME: The name of the app to stream logs from
- DRAIN-NAME: A name to use for your syslog drain service instance
- SYSLOG-DRAIN-URL: The syslog URL from [Step 1: Configure the Log Management Service](#step1).
After a short delay, logs begin to flow automatically.
Refer to the [Usage]( section of the CF Drain plugin source repository for CF Drain commands, and [Managing Service Instances with the CLI](./managing-services.html) for general CF service commands.
#### <a id='no-plugin'></a>With General cf CLI Service Commands
1. To create the service instance, run `cf create-user-provided-service` (or `cf cups`) with the `-l` flag, filling in values as follows:
- DRAIN-NAME: A name to use for your syslog drain service instance
- SYSLOG-DRAIN-URL: The syslog URL from [Step 1: Configure the Log Management Service](#step1).
<pre class="terminal">
$ cf create-user-provided-service DRAIN-NAME -l SYSLOG-URL
See [User-Provided Service Instances](./user-provided.html) for more information.
1. To bind an app to the service instance, do one of the following:
* Run `cf push` with a manifest. The services block in the manifest must specify the service instance that you want to bind.
* Run `cf bind-service`
<pre class="terminal">
$ cf bind-service YOUR-APP-NAME DRAIN-NAME
After a short delay, logs begin to flow automatically.
Refer to [Managing Service Instances with the CLI](./managing-services.html) for more information.
### <a id='cups-step3'></a>Step 3: Verify Logs Are Draining
To verify that logs are draining correctly to a third-party log management service:
1. Take actions that produce log messages, such as making requests of your app.
1. Compare the logs displayed in the CLI against those displayed by the log management service.
For example, if your application serves web pages, you can send HTTP requests to the application.
In Cloud Foundry, these generate Router log messages, which you can view in the CLI.
Your third-party log management service should display corresponding messages.
<p class="note"><strong>Note</strong>: For security reasons, Cloud Foundry applications do not respond to <code>ping</code>. You cannot use <code>ping</code> to generate log entries.</p>
### <a id='drain-cli'></a>CF Drain CLI Plugin
The CF Drain CLI plugin extends the cf CLI by adding simple commands for user-provided syslog drains.
**Installation**: To install the CF Drain CLI plugin see the [Installing Plugin]( instructions in the plugin source repository.
**Commands**: The plugin adds commands for creating, deleting, and listing syslog drains, and for binding apps to the drains. See the [Usage]( section of the plugin source repository for details.