CF + ELK
Logging in the clouds

As a CF operator
I can extend my Cloud Foundry platform with log analysis
Using Logsearch for Cloudfoundry

David Laing Product Owner, Logsearch

Jack Wu (吴疆) Cloud Ops, Pivotal

---

Microservices - must be this tall

• Operating distributed systems is hard
  • Cloud Foundry platform
  • Microservices running on Cloud Foundry
• Centralised log and metric analysis lets you see what is going on
• Should be part of the platform

---

Logsearch - adding ELK to CF platform

• ELK stack deployed by BOSH
  • Running in production for 2 years
  • Log processing pipeline -> Log search engine -> Visualization tool
  • Linear scalability
• Logsearch for Cloud Foundry adds tight integration into CF ecosystem
  • Ingests all the logs ( and metrics )
  • Extracts and enriches
  • Zero configuration authentication and multitenant access control

---

Log sources

0. Platform component logs from syslog forwarders
1. Platform metrics from CF Collector
2. Deployment data from BOSH NATs
3. App logs from Loggregator firehose

---

Listen the Voice -- ELK for CloudFoundry

• Log sources
  • Syslog of CloudFoundry components
  • AWS ELB access log
  • AWS RDS Log
• High Avaiability
  • Multi-AZ deployment
• Scalability
  • storage - extend storage when necessary
  • computing - adding parser nodes when necessary

---

Multi-AZ ELK architecture

---

Multi-AZ ELK in Pivotal Webservice (PWS)

• Deployment size
  • Totally 37 nodes among 3 Available Zones (AZ)
  • Totally 9 parser nodes (c3.large VM)
  • Totally 15 Elastic Search Nodes (r3.xlarge nodes)
  • Totally storage size - 18T
• Data size
  • 18 million lines log per hour
  • 27G indexed data per hour

---

Listening PWS with ELK - 1

---

Listening PWS with ELK - 2

---

Listening PWS with ELK - 3

---

DEMO: BOSH Health Monitor Metrics

---

DEMO: Enhanced Blue / Green deployment

---

ELK for PCF tile

• Tile for Pivotal's Cloud Foundry distribution
• Adds
  • Support for Spring Cloud Microservices
  • Focus on security and audit logs
  • Commercial support

---

DEMO: Cross Microservice Transaction Tracing

---

Wrap-up

• Log analysis enables operating distributed systems
• Logsearch for Cloud Foundry
    -> logs analysis for CF platform
• ELK for PCF tile
    -> commercial support with focus on Spring Cloud