Permalink
Browse files

moar slides

  • Loading branch information...
1 parent 21d1249 commit 76f908528e0a541be8389087fe7db9c04065dd99 @jordansissel committed Aug 12, 2012
View
20 presentations/logs-and-logstash/about-me/slides.md
@@ -0,0 +1,20 @@
+!SLIDE transition=fade
+# who am i?
+
+* sysadmin
+* coder
+* dad
+* beer and rum!
+
+!SLIDE transition=fade center
+![tequila face](tequila-face.jpg)
+
+no tequila plz
+
+!SLIDE transition=fade
+# other fun projects
+
+* fpm
+* xdotool
+* fingerpoken
+* fex
View
BIN presentations/logs-and-logstash/about-me/tequila-face.jpg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
26 presentations/logs-and-logstash/grok/slides.md
@@ -52,6 +52,30 @@ why do developers keep writing crappy log formats?
* It's easy to add new ones.
!SLIDE transition=fade incremental
-# grok : apache
+# grok discovery
+Logs -> Patterns for those logs
+!SLIDE transition=fade incremental
+# grok discovery
+
+* Apr 20 00:53:46 rickastley roll: Never gonna give you up.
+* %{SYSLOGBASE}\Q Never gonna give you up.\E
+
+!SLIDE transition=fade incremental
+
+%{SYSLOGBASE}\Q Never gonna give you up.\E
+
+<pre style="word-wrap: break-word; font-size: 2em">
+\Q\E(?<0000>(?<0001>(?<0002>\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b) +(?<0003>(?:3[01]|[1-2]?[0-9]|0?[1-9])) (?<0004>(?!<[0-9])(?<0005>(?:2[0123]|[01][0-9])):(?<0006>(?:[0-5][0-9]))(?::(?<0007>(?:(?:[0-5][0-9]|60)(?:[.,][0-9]+)?)))(?![0-9]))) (?:(?<0008><(?<0009>\b(?:[0-9]+)\b).(?<000a>\b(?:[0-9]+)\b)>) )?(?<000b>(?<000c>(?:(?<000d>\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b))|(?<000e>(?<![0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9]))))) (?<000f>(?<0010>(?:[\w._/-]+))(?:\[(?<0011>\b(?:[0-9]+)\b)\])?):)\Q Never gonna give you up.\E
+</pre>
+
+!SLIDE transition=fade incremental
+
+input:
+
+* Aug 23 12:04:33 "hello world" 123.4.3.5 something something woo!
+
+output:
+
+* `%{SYSLOGTIMESTAMP} %{QS} %{IP} something something woo!`
View
25 presentations/logs-and-logstash/intro-logging-problems/slides.md
@@ -210,11 +210,11 @@ Isn't this better than reading raw logs?
* Ship logs away from edge/application servers
!SLIDE transition=fade bullets incremental
-# Expertise Necessary
+# Knowledge Gap
-* Knowing what question to ask
+* What question to ask?
* vs
-* Knowing how to answer the question
+* How to answer the question?
!SLIDE transition=fade center
@@ -224,6 +224,8 @@ Isn't this better than reading raw logs?
You'll be a hero.
+_xkcd #208_
+
!SLIDE transition=fade center
.notes Except now, any time those two folks want questions answered, they'll ask you.
@@ -290,7 +292,7 @@ seriously.
* _except when it's not_
-!SLIDE transition=fade full-screen
+!SLIDE transition=fade full-screen nowrap
# one event
<pre style="font-size: 2em">
@@ -310,3 +312,18 @@ org.omg.CORBA.MARSHAL: com.ibm.ws.pmi.server.DataDescriptor; IllegalAccessExcept
</pre>
_(logstash solves this one easy)_
+
+!SLIDE transition=fade incremental
+# 'log reference guides'
+
+* NetScreen Log Guide: 170 pages
+* FortiGate Log Guide: 80 pages
+* SonicOS Log Guide: 122 pages
+* ProCurve Log Guide: 56 pages
+
+
+!SLIDE transition=fade incremental
+# 'log reference guides'
+
+* Probability these guides are accurate: 0%
+* Reinforces antipattern of reading each event by a human.
View
5 presentations/logs-and-logstash/life-of-a-log/slides.md
@@ -14,19 +14,20 @@ emit | transport | analyze | store
* Ship application logs somewhere
* Possibly anonymize them in transit.
-* syslog, scribe, flume, etc
+* logstash, syslog, scribe, flume, etc
!SLIDE transition=fade
# Analytics
* Search and Analysis
-* Tools: Hadoop, ElasticSearch, Graphite, etc
+* Tools: logstash, Hadoop, ElasticSearch, Graphite
!SLIDE transition=fade
# Storage
* HDFS, S3, Sherpa, etc.
* How much can you store?
* How much do you need to store?
+* What's your retention policy?
View
20 presentations/logs-and-logstash/logstash-about/slides.md
@@ -3,10 +3,17 @@
as an open source project
-!SLIDE transition=fade
+!SLIDE transition=fade incremental
+# Project Rules
+
+* If a newbie has a bad time, it's a bug. Period.
+* If it seems awkward, it probably is.
+* If it's not possible, we can make it possible.
+* All contributions are good contributions.
+
+!SLIDE transition=fade incremental
# Community
-* Everyone can contribute
* Focus on reducing cost-to-implement
* Apache 2.0 license
@@ -37,3 +44,12 @@ as an open source project
* 2000+ deployments
* Many with > 10,000 events/sec
* Many with > 1TB/day
+
+!SLIDE transition=fade
+# Community Support
+
+* Site: <http://logstash.net>
+* IRC: #logstash on freenode
+* Email: logstash-users@googlegroups.com
+* Tickets: <http://logstash.jira.com>
+* Code: <https://github.com/logstash/logstash>
View
BIN presentations/logs-and-logstash/logstash-roadmap/elasticsearch-logstash-piesnacking.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
19 presentations/logs-and-logstash/logstash-roadmap/slides.md
@@ -0,0 +1,19 @@
+!SLIDE transition=fade incremental
+# logstash roadmap
+
+!SLIDE transition=fade incremental
+# search and analytics
+
+!SLIDE transition=fade center
+# improve kibana
+
+!SLIDE transition=fade center
+# vxin
+![vxin](elasticsearch-logstash-piesnacking.png)
+
+!SLIDE transition=fade incremental
+# cost of operation
+
+!SLIDE transition=fade incremental
+# release frequency
+
View
BIN presentations/logs-and-logstash/logstash/kibana.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
BIN presentations/logs-and-logstash/logstash/log-management.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
BIN presentations/logs-and-logstash/logstash/multi-host-example.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
BIN presentations/logs-and-logstash/logstash/single-host-example.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
34 presentations/logs-and-logstash/logstash/slides.md
@@ -7,13 +7,18 @@ So how does logstash fit in?
.notes TODO
!SLIDE transition=fade incremental
-# Goals
+# Goals: Log Life Cycle
-* Tooling for managing log lifecycle
* Take events.
* Massage them.
* Put them somewhere else.
-* Don't be annoying.
+
+!SLIDE transition=fade incremental
+# Goals: User Experience
+
+* Fit your infrastructure
+* Be extentable
+* Be well documented
!SLIDE transition=fade incremental
# logstash agent
@@ -53,8 +58,16 @@ inputs | filters | outputs
* /var/log/*.log (file input)
* grok filter (parse said logs)
+* date filter (normalize the date)
* elasticsearch output (for storage/search/analytics)
-* graphite output (for metrics/trending)
+
+!SLIDE transition=fade center fullwidth
+![single node example](single-host-example.png)
+## one agent, one server
+
+!SLIDE transition=fade center fullwidth
+![multi node example](multi-host-example.png)
+## many nodes, tiered deployment
!SLIDE transition=fade incremental
# common case
@@ -63,3 +76,16 @@ inputs | filters | outputs
* logstash slurps them up
* ships to elasticsearch
* search/analytics with elasticsearch
+
+!SLIDE transition=fade incremental
+# logstash agent - transport
+
+* A few plugins are for transporting logs
+* redis, amqp, stomp, tcp, zeromq, jabber, irc, syslog
+* This lets you pipe two remote logstash agents together
+
+!SLIDE transition=fade incremental
+# logstash analytics
+
+TBD: kibana screenshots
+
View
2 presentations/logs-and-logstash/showoff.json
@@ -2,12 +2,14 @@
"name": "logs and logstash",
"sections": [
{ "section": "title" },
+ { "section": "about-me" },
{ "section": "intro-logging-problems" },
{ "section": "terminology" },
{ "section": "life-of-a-log" },
{ "section": "logstash" },
{ "section": "grok" },
{ "section": "use-cases" },
+ { "section": "logstash-roadmap" },
{ "section": "logstash-about" }
]
}
View
11 presentations/logs-and-logstash/style.css
@@ -23,6 +23,17 @@ pre {
word-wrap: break-word;
}
+.nowrap pre {
+ word-wrap: normal;
+ white-space: pre;
+}
+
+
.smallpre pre {
font-size: 250%;
}
+
+
+.fullwidth img {
+ width: 100%;
+}

0 comments on commit 76f9085

Please sign in to comment.