moar slides

presentations/logs-and-logstash/about-me/slides.md

@@ -0,0 +1,20 @@
+!SLIDE transition=fade  
+# who am i?
+
+* sysadmin
+* coder
+* dad
+* beer and rum!
+
+!SLIDE transition=fade center
+![tequila face](tequila-face.jpg)
+
+no tequila plz
+
+!SLIDE transition=fade
+# other fun projects
+
+* fpm
+* xdotool
+* fingerpoken
+* fex

presentations/logs-and-logstash/grok/slides.md

@@ -52,6 +52,30 @@ why do developers keep writing crappy log formats?
 * It's easy to add new ones.
 
 !SLIDE transition=fade incremental
-# grok : apache
+# grok discovery
 
+Logs -> Patterns for those logs
 
+!SLIDE transition=fade incremental
+# grok discovery
+
+* Apr 20 00:53:46 rickastley roll: Never gonna give you up.
+* %{SYSLOGBASE}\Q Never gonna give you up.\E
+
+!SLIDE transition=fade incremental
+
+%{SYSLOGBASE}\Q Never gonna give you up.\E
+
+<pre style="word-wrap: break-word; font-size: 2em">
+\Q\E(?<0000>(?<0001>(?<0002>\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b) +(?<0003>(?:3[01]|[1-2]?[0-9]|0?[1-9])) (?<0004>(?!<[0-9])(?<0005>(?:2[0123]|[01][0-9])):(?<0006>(?:[0-5][0-9]))(?::(?<0007>(?:(?:[0-5][0-9]|60)(?:[.,][0-9]+)?)))(?![0-9]))) (?:(?<0008><(?<0009>\b(?:[0-9]+)\b).(?<000a>\b(?:[0-9]+)\b)>) )?(?<000b>(?<000c>(?:(?<000d>\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b))|(?<000e>(?<![0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9]))))) (?<000f>(?<0010>(?:[\w._/-]+))(?:\[(?<0011>\b(?:[0-9]+)\b)\])?):)\Q Never gonna give you up.\E
+</pre>
+
+!SLIDE transition=fade incremental
+
+input:
+
+* Aug 23 12:04:33 "hello world" 123.4.3.5 something something woo!
+
+output:
+
+* `%{SYSLOGTIMESTAMP} %{QS} %{IP} something something woo!`

presentations/logs-and-logstash/intro-logging-problems/slides.md

@@ -210,11 +210,11 @@ Isn't this better than reading raw logs?
 * Ship logs away from edge/application servers
 
 !SLIDE transition=fade bullets incremental
-# Expertise Necessary 
+# Knowledge Gap
 
-* Knowing what question to ask
+* What question to ask?
 * vs
-* Knowing how to answer the question
+* How to answer the question?
 
 !SLIDE transition=fade center
 
@@ -224,6 +224,8 @@ Isn't this better than reading raw logs?
 
 You'll be a hero.
 
+_xkcd #208_
+
 !SLIDE transition=fade center
 
 .notes Except now, any time those two folks want questions answered, they'll ask you.
@@ -290,7 +292,7 @@ seriously.
 
 * _except when it's not_
 
-!SLIDE transition=fade full-screen
+!SLIDE transition=fade full-screen nowrap
 # one event
 
 <pre style="font-size: 2em">
@@ -310,3 +312,18 @@ org.omg.CORBA.MARSHAL: com.ibm.ws.pmi.server.DataDescriptor; IllegalAccessExcept
 </pre>
 
 _(logstash solves this one easy)_
+
+!SLIDE transition=fade incremental
+# 'log reference guides'
+
+* NetScreen Log Guide: 170 pages
+* FortiGate Log Guide: 80 pages
+* SonicOS Log Guide: 122 pages
+* ProCurve Log Guide: 56 pages
+
+
+!SLIDE transition=fade incremental
+# 'log reference guides'
+
+* Probability these guides are accurate: 0%
+* Reinforces antipattern of reading each event by a human.

presentations/logs-and-logstash/life-of-a-log/slides.md

@@ -14,19 +14,20 @@ emit | transport | analyze | store
 
 * Ship application logs somewhere
 * Possibly anonymize them in transit.
-* syslog, scribe, flume, etc
+* logstash, syslog, scribe, flume, etc
 
 !SLIDE transition=fade
 # Analytics
 
 * Search and Analysis
-* Tools: Hadoop, ElasticSearch, Graphite, etc
+* Tools: logstash, Hadoop, ElasticSearch, Graphite
 
 !SLIDE transition=fade
 # Storage
 
 * HDFS, S3, Sherpa, etc.
 * How much can you store?
 * How much do you need to store?
+* What's your retention policy?
 
 

presentations/logs-and-logstash/logstash-about/slides.md

@@ -3,10 +3,17 @@
 
 as an open source project
 
-!SLIDE transition=fade
+!SLIDE transition=fade incremental
+# Project Rules
+
+* If a newbie has a bad time, it's a bug. Period.
+* If it seems awkward, it probably is.
+* If it's not possible, we can make it possible.
+* All contributions are good contributions.
+
+!SLIDE transition=fade incremental
 # Community
 
-* Everyone can contribute
 * Focus on reducing cost-to-implement
 * Apache 2.0 license
 
@@ -37,3 +44,12 @@ as an open source project
 * 2000+ deployments
 * Many with > 10,000 events/sec
 * Many with > 1TB/day
+
+!SLIDE transition=fade
+# Community Support
+
+* Site: <http://logstash.net>
+* IRC: #logstash on freenode
+* Email: logstash-users@googlegroups.com
+* Tickets: <http://logstash.jira.com>
+* Code: <https://github.com/logstash/logstash>

presentations/logs-and-logstash/logstash-roadmap/slides.md

@@ -0,0 +1,19 @@
+!SLIDE transition=fade incremental
+# logstash roadmap
+
+!SLIDE transition=fade incremental
+# search and analytics
+
+!SLIDE transition=fade center
+# improve kibana
+
+!SLIDE transition=fade center
+# vxin
+![vxin](elasticsearch-logstash-piesnacking.png)
+
+!SLIDE transition=fade incremental
+# cost of operation
+
+!SLIDE transition=fade incremental
+# release frequency
+

presentations/logs-and-logstash/logstash/slides.md

@@ -7,13 +7,18 @@ So how does logstash fit in?
 .notes TODO
 
 !SLIDE transition=fade incremental
-# Goals
+# Goals: Log Life Cycle
 
-* Tooling for managing log lifecycle
 * Take events.
 * Massage them.
 * Put them somewhere else.
-* Don't be annoying.
+
+!SLIDE transition=fade incremental
+# Goals: User Experience
+
+* Fit your infrastructure
+* Be extentable
+* Be well documented
 
 !SLIDE transition=fade incremental
 # logstash agent
@@ -53,8 +58,16 @@ inputs | filters | outputs
 
 * /var/log/*.log (file input)
 * grok filter (parse said logs)
+* date filter (normalize the date)
 * elasticsearch output (for storage/search/analytics)
-* graphite output (for metrics/trending)
+
+!SLIDE transition=fade center fullwidth
+![single node example](single-host-example.png)
+## one agent, one server
+
+!SLIDE transition=fade center fullwidth
+![multi node example](multi-host-example.png)
+## many nodes, tiered deployment
 
 !SLIDE transition=fade incremental
 # common case
@@ -63,3 +76,16 @@ inputs | filters | outputs
 * logstash slurps them up
 * ships to elasticsearch
 * search/analytics with elasticsearch
+
+!SLIDE transition=fade incremental
+# logstash agent - transport
+
+* A few plugins are for transporting logs
+* redis, amqp, stomp, tcp, zeromq, jabber, irc, syslog
+* This lets you pipe two remote logstash agents together
+
+!SLIDE transition=fade incremental
+# logstash analytics 
+
+TBD: kibana screenshots
+

presentations/logs-and-logstash/showoff.json

@@ -2,12 +2,14 @@
   "name": "logs and logstash", 
   "sections": [ 
     { "section": "title" },
+    { "section": "about-me" },
     { "section": "intro-logging-problems" },
     { "section": "terminology" },
     { "section": "life-of-a-log" },
     { "section": "logstash" },
     { "section": "grok" },
     { "section": "use-cases" },
+    { "section": "logstash-roadmap" },
     { "section": "logstash-about" }
   ]
 }

presentations/logs-and-logstash/style.css

@@ -23,6 +23,17 @@ pre {
   word-wrap: break-word;
 }
 
+.nowrap pre {
+  word-wrap: normal;
+  white-space: pre;
+}
+
+
 .smallpre pre {
   font-size: 250%;
 }
+
+
+.fullwidth img {
+  width: 100%;
+}