Loggregator

Logging in the Clouds

Loggregator is the user application logging subsystem for Cloud Foundry.

Features

Loggregator allows users to:

1. Tail their application logs.
2. Dump a recent set of application logs (where recent is on the order of an hour).
3. Continually drain their application logs to 3rd party log archive and analysis services.

Usage

cf logs [--app=APP_NAME] [--recent]

APP_NAME defaults to the current application in your manifest.yml file.

$ cf logs
Started GET "/" for 127.0.0.1 at 2013-04-05 13:14:58 -0700
Processing by Rails::WelcomeController#index as HTML
  Rendered /Users/tsaleh/.rbenv/versions/2.0.0-p0/lib/ruby/gems/2.0.0/gems/railties-4.0.0.beta1/lib/rails/templates/rails/welcome/index.html.erb (1.9ms)
Completed 200 OK in 12ms (Views: 11.3ms | ActiveRecord: 0.0ms)
Started GET "/assets/rails.png" for 127.0.0.1 at 2013-04-05 13:14:58 -0700
...
^C

Constraints

1. Loggregator collects STDOUT & STDERR from the customer's application. This may require configuration on the developer's side.
2. A Loggregator outage must not affect the running application.
3. Loggregator gathers and stores logs in a best-effort manner. While undesirable, losing the current buffer of application logs is acceptable.
4. The 3rd party drain API should mimic Heroku's in order to reduce integration effort for our partners. The Heroku drain API is simply remote syslog over TCP.

Architecture

Loggregator is composed of:

• Sources: Logging agents that run on the Cloud Foundry components. They forward logs to:
• Loggregator Server: Responsible for gathering logs from the sources, and storing in the temporary buffers.
• Hashing Layer: Makes the Loggregator Servers horizontally scalable by partitioning incoming log messages and outgoing traffic. Routes incoming log messages and proxies outgoing connections to the CLI and to drains for 3rd party partners.

Source agents emit the logging data as protocol-buffers, and the data stays in that format throughout the system.

Emitting Messages from other Cloud Foundry components

Cloud Foundry developers can easily add source clients to new CF components that emit messages to the loggregator server. Currently, there are libraries for Go and Ruby. For usage information, look at their respective READMEs.

Development

The Cloud Foundry team uses GitHub and accepts contributions via pull request.

Follow these steps to make a contribution to any of our open source repositories:

1. Complete our CLA Agreement for individuals or corporations

2. Set your name and email

   git config --global user.name "Firstname Lastname"
   git config --global user.email "your_email@youremail.com"
   

3. Fork the repo

4. Make your changes on a topic branch, commit, and push to github and open a pull request.

Once your commits are approved by Travis CI and reviewed by the core team, they will be merged.

Installing Go on OS X

Use brew and do

brew install go --cross-compile-all

Checkout

git clone https://github.com/cloudfoundry/loggregator
cd loggregator
git submodule update --init

Running tests

bin/test

Debugging

Loggregator will dump information about the running goroutines to stdout if sent a USR1 signal.

goroutine 1 [running]:
runtime/pprof.writeGoroutineStacks(0xc2000bc3f0, 0xc200000008, 0xc200000001, 0xca0000c2001fcfc0)
	/home/travis/.gvm/gos/go1.1.1/src/pkg/runtime/pprof/pprof.go:511 +0x7a
runtime/pprof.writeGoroutine(0xc2000bc3f0, 0xc200000008, 0x2, 0xca74765c960d5c8f, 0x40bbf7, ...)
	/home/travis/.gvm/gos/go1.1.1/src/pkg/runtime/pprof/pprof.go:500 +0x3a
....

Development binary builds

Note: This is linux amd64 only

• loggregator