Skip to content
Browse files

refactored clarity to work as gem

  • Loading branch information...
1 parent 2b4f1b1 commit 17b3f58e34f8d603979c6203195ffa7077ddb6b4 @tobi committed Oct 31, 2009
View
4 History.txt
@@ -0,0 +1,4 @@
+=== 0.0.1 2009-10-31
+
+* 1 major enhancement:
+ * Initial release
View
89 Manifest.txt
@@ -0,0 +1,89 @@
+.git/COMMIT_EDITMSG
+.git/HEAD
+.git/config
+.git/description
+.git/hooks/applypatch-msg.sample
+.git/hooks/commit-msg.sample
+.git/hooks/post-commit.sample
+.git/hooks/post-receive.sample
+.git/hooks/post-update.sample
+.git/hooks/pre-applypatch.sample
+.git/hooks/pre-commit.sample
+.git/hooks/pre-rebase.sample
+.git/hooks/prepare-commit-msg.sample
+.git/hooks/update.sample
+.git/index
+.git/info/exclude
+.git/logs/HEAD
+.git/logs/refs/heads/master
+.git/logs/refs/remotes/origin/HEAD
+.git/objects/0a/e16403b3e558d00643f004ec09f334d4f2b0f0
+.git/objects/13/d34b561193da18ea9e4c23baa2889925b136c2
+.git/objects/14/7cadaa8ea2f5b519d51d463a054506d3fa2daa
+.git/objects/2b/4f1b1c8f1e8f29808e84c1253489f9cc6e9b23
+.git/objects/2f/2da91b54af9c05d7411ecc1c39c1617f1f4043
+.git/objects/3f/7f20971498dc181ee1df392377962dd9cb0ac3
+.git/objects/51/bf54fff55b5435bec597ec2c53c60331791f2d
+.git/objects/55/9c5567b25ddf52539d7cd4d174499fe1245fc0
+.git/objects/5c/4f9e74658278d187438bcdf769d3f47f306aab
+.git/objects/68/80b0fe88669a8b674a51e942e081c456a87198
+.git/objects/69/cfadbf94e845e48bfc06ec546cc66f9774eff4
+.git/objects/6a/afe1103b308a08e138124dd3dc2ba8f6229aae
+.git/objects/6f/a67dd445c01358acb9e4f65ed968a04df9aeaf
+.git/objects/72/348477a1ef597f318f3481a4021b9c27c93e1f
+.git/objects/81/51694f42a34ffeabf9bebaa7ef6f0dc85666e9
+.git/objects/89/35f726967181bf0a8dd776a8a643587d225491
+.git/objects/89/478d30f5c28c6a505f27a5ba0846e04aef4196
+.git/objects/93/a0697eeb2ebf39059ea24a3f94d6896b0a738e
+.git/objects/94/8081d177ae3347417a8911eae102f5d453eea0
+.git/objects/b7/69625e57eb1a1bad591f602fc874af1517febf
+.git/objects/c2/7f6559350f7adb19d43742b55b2f91d07b6550
+.git/objects/c7/15d3c368c22eba9f6b891cdb48ca02d9aacc9a
+.git/objects/cd/7ccf92f1c2dd5232f70fa5a03ad8eefb223e38
+.git/objects/d4/5ac4d0c9af69f25101640eff55bdd004dca4e9
+.git/objects/e4/8464df56bf487e96e21ea99487330266dae3c9
+.git/objects/e5/49cee6229b3c814e7b233643d3782b79753f0f
+.git/objects/f4/15af7e70bbed51b80762bd47df0c61d5e530f1
+.git/objects/pack/pack-624e8a418388f1f608033bc75af8982ce8e09c4d.idx
+.git/objects/pack/pack-624e8a418388f1f608033bc75af8982ce8e09c4d.pack
+.git/packed-refs
+.git/refs/heads/master
+.git/refs/remotes/origin/HEAD
+.gitignore
+History.txt
+Manifest.txt
+PostInstall.txt
+README.rdoc
+Rakefile
+bin/clarity
+config/config.yml.sample
+lib/clarity.rb
+lib/clarity/cli.rb
+lib/clarity/commands/command_builder.rb
+lib/clarity/commands/tail_command_builder.rb
+lib/clarity/parsers/hostname_parser.rb
+lib/clarity/parsers/shop_parser.rb
+lib/clarity/parsers/time_parser.rb
+lib/clarity/renderers/log_renderer.rb
+lib/clarity/server.rb
+lib/clarity/server/basic_auth.rb
+lib/clarity/server/chunk_http.rb
+lib/clarity/server/mime_types.rb
+public/images/spinner_big.gif
+public/javascripts/app.js
+public/stylesheets/app.css
+script/console
+script/destroy
+script/generate
+test/commands/command_builder_test.rb
+test/commands/tail_command_builder_test.rb
+test/files/logfile.log
+test/parsers/hostname_parser_test.rb
+test/parsers/shop_parser_test.rb
+test/parsers/time_parser_test.rb
+test/test_helper.rb
+test/test_string_scanner.rb
+views/_header.html.erb
+views/_toolbar.html.erb
+views/error.html.erb
+views/index.html.erb
View
8 PostInstall.txt
@@ -0,0 +1,8 @@
+
+For more information on clarity, see http://github/tobi/clarity
+
+You can try clarity by running:
+ clarity -p 3000 /var/log
+
+
+
View
56 README.rdoc
@@ -0,0 +1,56 @@
+= Clarity
+
+* http://github.com/#{github_username}/#{project_name}
+
+== DESCRIPTION:
+
+Clarity - a log search tool
+By John Tajima & Tobi Lütke
+
+Clarity is an eventmachine-based web application that is used at Shopify to
+search log files on production servers.
+
+We wrote Clarity to allow authorized users to use a simple interface to look
+through the various log files in our server farm, without requiring access to
+production servers.
+
+Clarity requires eventmachine and eventmachine/evma_httpserver.
+
+ sudo gem install eventmachine eventmachine_httpserver
+
+
+== REQUIREMENTS:
+
+* eventmachine
+* eventmachine_httpserver
+
+== INSTALL:
+
+* sudo gem install clarity
+
+== LICENSE:
+
+(The MIT License)
+
+Copyright (c) 2009 Tobias Lütke
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OT
+
+
View
29 README.txt
@@ -1,29 +0,0 @@
-Clarity - a log search tool
-By John Tajima & Tobi Lütke
----------------------------------------------------------------------------------
-
-Clarity is an eventmachine-based web application that is used at Shopify to
-search log files on production servers.
-
-We wrote Clarity to allow authorized users to use a simple interface to look
-through the various log files in our server farm, without requiring access to
-production servers.
-
-Clarity requires eventmachine and eventmachine/evma_httpserver.
-
- sudo gem install eventmachine eventmachine_httpserver
-
-
-
-Changelog
----------
-
-Oct 31, 2009 - Added command line interface
-
-
-
-Sept 12, 2009 - Search terms are now optional. If no search terms are submitted,
- uses (gz)cat instead of (z)grep or tail
- - Refactoring of commands, parsers, view templates
- - Added tests
-
View
41 Rakefile
@@ -1,14 +1,27 @@
-require 'rake'
-require 'rake/testtask'
-require 'rake/rdoctask'
-
-desc 'Default: run unit tests.'
-task :default => :test
-
-desc 'Test the http_auth plugin.'
-Rake::TestTask.new(:test) do |t|
- t.libs << 'lib'
- t.pattern = 'test/**/*_test.rb'
- t.verbose = true
-end
-
+require 'rubygems'
+gem 'hoe', '>= 2.1.0'
+require 'hoe'
+require 'fileutils'
+require './lib/clarity'
+
+Hoe.plugin :newgem
+# Hoe.plugin :website
+# Hoe.plugin :cucumberfeatures
+
+# Generate all the Rake tasks
+# Run 'rake -T' to see list of generated tasks (from gem root directory)
+$hoe = Hoe.spec 'clarity' do
+ self.developer 'Tobias Lütke', 'tobi@shopify.com'
+ self.developer 'John Tajima', 'john@shopify.com'
+ self.post_install_message = 'PostInstall.txt' # TODO remove if post-install message not required
+ # self.rubyforge_name = self.name # TODO this is default value
+
+ self.extra_deps = [['eventmachine','>= 0.12.10'], ['eventmachine_httpserver','>= 0.2.0']]
+end
+
+require 'newgem/tasks'
+Dir['tasks/**/*.rake'].each { |t| load t }
+
+# TODO - want other tests/tasks run by default? Add them to the list
+# remove_task :default
+# task :default => [:spec, :features]
View
10 bin/clarity
@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+#
+# Created on 2009-10-31.
+# Copyright (c) 2009. All rights reserved.
+
+require 'rubygems'
+require File.expand_path(File.dirname(__FILE__) + "/../lib/clarity")
+require "clarity/cli"
+
+Clarity::CLI.execute(STDOUT, ARGV)
View
22 lib/basic_auth.rb
@@ -1,22 +0,0 @@
-module BasicAuth
-
- def decode_credentials(request)
- Base64.decode64(request).split.last
- end
-
- def user_name_and_password(request)
- decode_credentials(request).split(/:/, 2)
- end
-
- def authentication_data
- headers = @http_headers.split("\000")
- auth_header = headers.detect {|head| head =~ /Authorization: / }
- header = auth_header.nil? ? "" : auth_header.split("Authorization: Basic ").last
- return (user_name_and_password(header) rescue ['', ''])
- end
-
- def authenticate!(http_header)
- raise NotAuthenticatedError unless authenticate(http_header)
- end
-
-end
View
20 lib/clarity.rb
@@ -0,0 +1,20 @@
+$:.unshift(File.dirname(__FILE__)) unless $:.include?(File.dirname(__FILE__)) || $:.include?(File.expand_path(File.dirname(__FILE__)))
+
+require 'eventmachine'
+require 'evma_httpserver'
+require 'yaml'
+require 'base64'
+require 'clarity/server'
+require 'clarity/commands/command_builder'
+require 'clarity/commands/tail_command_builder'
+require 'clarity/parsers/time_parser'
+require 'clarity/parsers/hostname_parser'
+require 'clarity/parsers/shop_parser'
+require 'clarity/renderers/log_renderer'
+
+module Clarity
+ VERSION = '1.0.0'
+
+ Templates = File.dirname(__FILE__) + '/../views'
+ Public = File.dirname(__FILE__) + '/../public'
+end
View
118 lib/clarity/cli.rb
@@ -0,0 +1,118 @@
+require 'optparse'
+#require File.dirname(__FILE__) + '/../clarity'
+
+module GrepRenderer
+ attr_accessor :response, :parser, :marker, :params
+
+ def parser
+ @parser ||= TimeParser.new( HostnameParser.new(ShopParser.new), params)
+ end
+
+ def renderer
+ @renderer ||= LogRenderer.new
+ end
+
+ # once download is complete, send it to client
+ def receive_data(data)
+ @buffer ||= StringScanner.new("")
+ @buffer << data
+
+ html = ""
+ while line = @buffer.scan_until(/\n/)
+ tokens = parser.parse(line)
+ html << renderer.render(tokens)
+ end
+
+ return if html.empty?
+
+ response.chunk html
+ response.send_chunks
+ end
+
+ def unbind
+ response.chunk '</div><hr><p id="done">Done</p></body></html>'
+ response.chunk ''
+ response.send_chunks
+ puts 'Done'
+ end
+end
+
+
+
+
+module Clarity
+ class CLI
+ def self.execute(stdout, arguments=[])
+
+ options = {
+ :username => nil,
+ :password => nil,
+ :log_files => ['**/*.log*'],
+ :port => 8080,
+ :address => "0.0.0.0"
+ }
+
+ mandatory_options = %w( )
+
+ ARGV.options do |opts|
+ opts.banner = "Usage: #{File.basename($PROGRAM_NAME)} [options] directory"
+
+ opts.separator " "
+ opts.separator "Specific options:"
+
+ opts.on( "-f", "--config=FILE", String, "Config file (yml)" ) do |opt|
+ options.update YAML.load_file( opt )
+ end
+
+ opts.on( "-p", "--port=PORT", Integer, "Port to listen on" ) do |opt|
+ options[:port] = opt
+ end
+
+ opts.on( "-b", "--address=ADDRESS", String, "Address to bind to (default 0.0.0.0)" ) do |opt|
+ options[:address] = opt
+ end
+
+ opts.on( "--include=MASK", String, "File mask of logs to add (default: **/*.log*)" ) do |opt|
+ options[:log_files] ||= []
+ options[:log_files] += opt
+ end
+
+ opts.separator " "
+ opts.separator "Password protection:"
+
+ opts.on( "--username=USER", String, "Enable httpauth username" ) do |opt|
+ options[:username] = opt
+ end
+
+ opts.on( "--password=PASS", String, "Enable httpauth password" ) do |opt|
+ options[:password] = opt
+ end
+
+ opts.separator " "
+ opts.separator "Misc:"
+
+ opts.on( "-h", "--help", "Show this message." ) do
+ puts opts
+ exit
+ end
+
+ opts.separator " "
+
+ begin
+ opts.parse!(arguments)
+
+ if arguments.first
+ Dir.chdir(arguments.first)
+ end
+
+ ::Clarity::Server.run(options)
+
+ #rescue
+ # puts opts
+ # exit
+ end
+ end
+
+ end
+ end
+end
View
0 lib/commands/command_builder.rb → lib/clarity/commands/command_builder.rb
File renamed without changes.
View
0 lib/commands/tail_command_builder.rb → lib/clarity/commands/tail_command_builder.rb
File renamed without changes.
View
0 lib/parsers/hostname_parser.rb → lib/clarity/parsers/hostname_parser.rb
File renamed without changes.
View
0 lib/parsers/shop_parser.rb → lib/clarity/parsers/shop_parser.rb
File renamed without changes.
View
0 lib/parsers/time_parser.rb → lib/clarity/parsers/time_parser.rb
File renamed without changes.
View
0 lib/renderers/log_renderer.rb → lib/clarity/renderers/log_renderer.rb
File renamed without changes.
View
142 lib/clarity/server.rb
@@ -0,0 +1,142 @@
+require 'cgi'
+require File.dirname(__FILE__) + '/server/basic_auth'
+require File.dirname(__FILE__) + '/server/mime_types'
+require File.dirname(__FILE__) + '/server/chunk_http'
+
+module Clarity
+ class NotFoundError < StandardError; end
+ class NotAuthenticatedError < StandardError; end
+ class InvalidParameterError < StandardError; end
+
+ module Server
+ include EventMachine::HttpServer
+ include Clarity::BasicAuth
+ include Clarity::ChunkHttp
+
+ attr_accessor :required_username, :required_password
+ attr_accessor :log_files
+
+ def self.run(options)
+ EventMachine::run do
+ EventMachine.epoll
+ EventMachine::start_server(options[:address], options[:port], self) do |a|
+ a.log_files = options[:log_files]
+ end
+ puts "Listening #{options[:address]}:#{options[:port]}..."
+ puts "Adding log files: #{options[:log_files].inspect}"
+ end
+ end
+
+ def process_http_request
+ authenticate!
+
+ puts "action: #{path}"
+ puts "params: #{params.inspect}"
+
+ case path
+ when '/'
+ respond_with(200, welcome_page)
+
+ when '/perform'
+ if params.empty?
+ respond_with(200, welcome_page)
+ else
+ # get command
+ command = case params['tool']
+ when 'grep' then CommandBuilder.new(params).command
+ when 'tail' then TailCommandBuilder.new(params).command
+ else raise InvalidParameterError, "Invalid Tool parameter"
+ end
+ response = respond_with_chunks
+ response.chunk results_page # display page header
+
+ puts "Running: #{command}"
+ EventMachine::popen(command, GrepRenderer) do |grepper|
+ @grepper = grepper
+ @grepper.marker = 0
+ @grepper.params = params
+ @grepper.response = response
+ end
+ end
+
+ when '/test'
+ response = init_chunk_response
+ EventMachine::add_periodic_timer(1) do
+ response.chunk "Lorem ipsum dolor sit amet<br/>"
+ response.send_chunks
+ end
+
+ else
+ respond_with(200, public_file(path), :content_type => Mime.for(path))
+ end
+
+ rescue InvalidParameterError => e
+ respond_with(500, error_page(e))
+ rescue NotFoundError => e
+ respond_with(404, "<h1>Not Found</h1>")
+ rescue NotAuthenticatedError => e
+ puts "Could not authenticate user"
+ headers = { "WWW-Authenticate" => %(Basic realm="Clarity")}
+ respond_with(401, "HTTP Basic: Access denied.\n", :content_type => 'text/plain', :headers => headers)
+ end
+
+ def error_page(error)
+ @error = error
+ render "error.html.erb"
+ end
+
+ def welcome_page
+ render "index.html.erb"
+ end
+
+ def results_page
+ render "index.html.erb"
+ end
+
+ def unbind
+ return unless @grepper
+ kill_processes(@grepper.get_status.pid)
+ close_connection
+ end
+
+ def kill_processes(ppid)
+ return if ppid.nil?
+ all_pids = [ppid] + get_child_pids(ppid).flatten.uniq.compact
+ puts "=== pids are #{all_pids.inspect}"
+ all_pids.each do |pid|
+ Process.kill('TERM',pid.to_i)
+ puts "=== killing #{pid}"
+ end
+ rescue Exception => e
+ puts "!Error killing processes: #{e}"
+ end
+
+ def get_child_pids(ppid)
+ out = `ps -opid,ppid | grep #{ppid.to_s}`
+ ids = out.split("\n").map {|line| $1 if line =~ /^\s*([0-9]+)\s.*/ }.compact
+ ids.delete(ppid.to_s)
+ if ids.empty?
+ ids
+ else
+ ids << ids.map {|id| get_child_pids(id) }
+ end
+ end
+
+ private
+
+
+
+ def authenticate!
+ login, pass = authentication_data
+
+ if (required_username && required_username != login) || (required_password && required_password != pass)
+ raise NotAuthenticatedError
+ end
+
+ true
+ end
+
+ end
+
+
+end
View
24 lib/clarity/server/basic_auth.rb
@@ -0,0 +1,24 @@
+module Clarity
+ module BasicAuth
+
+ def decode_credentials(request)
+ Base64.decode64(request).split.last
+ end
+
+ def user_name_and_password(request)
+ decode_credentials(request).split(/:/, 2)
+ end
+
+ def authentication_data
+ headers = @http_headers.split("\000")
+ auth_header = headers.detect {|head| head =~ /Authorization: / }
+ header = auth_header.nil? ? "" : auth_header.split("Authorization: Basic ").last
+ return (user_name_and_password(header) rescue ['', ''])
+ end
+
+ def authenticate!(http_header)
+ raise NotAuthenticatedError unless authenticate(http_header)
+ end
+
+ end
+end
View
57 lib/clarity/server/chunk_http.rb
@@ -0,0 +1,57 @@
+module Clarity
+
+ module ChunkHttp
+
+ LeadIn = ' ' * 1024
+
+ def respond_with_chunks
+ response = EventMachine::DelegatedHttpResponse.new( self )
+ response.status = 200
+ response.headers['Content-Type'] = 'text/html'
+ response.chunk LeadIn
+ response
+ end
+
+ def respond_with(status, content, options = {})
+ response = EventMachine::DelegatedHttpResponse.new( self )
+ response.headers['Content-Type'] = options.fetch(:content_type, 'text/html')
+ response.headers['Cache-Control'] = 'private, max-age=0'
+ headers = options.fetch(:headers, {})
+ headers.each_pair {|h, v| response.headers[h] = v }
+ response.status = status
+ response.content = content
+ response.send_response
+ end
+
+ def render(view)
+ @toolbar = template("_toolbar.html.erb")
+ @content_for_header = template("_header.html.erb")
+ template(view)
+ end
+
+ def template(filename)
+ content = File.read( File.join(Clarity::Templates, filename) )
+ ERB.new(content).result(binding)
+ end
+
+ def public_file(filename)
+ File.read( File.join(Clarity::Public, filename) )
+ rescue Errno::ENOENT
+ raise NotFoundError
+ end
+
+ def logfiles
+ log_files.map {|f| Dir[f] }.flatten.compact.uniq.select{|f| File.file?(f) }.sort
+ end
+
+ def params
+ ENV['QUERY_STRING'].split('&').inject({}) {|p,s| k,v = s.split('=');p[k.to_s] = CGI.unescape(v.to_s);p}
+ end
+
+ def path
+ ENV["PATH_INFO"]
+ end
+
+ end
+
+end
View
23 lib/clarity/server/mime_types.rb
@@ -0,0 +1,23 @@
+module Clarity
+ module Mime
+ def self.for(filename)
+
+ content_type = TYPES[File.extname(filename)]
+ content_type || 'text/plain'
+ end
+
+ TYPES = {
+ '.jpg' => 'image/jpg',
+ '.jpeg' => 'image/jpeg',
+ '.gif' => 'image/gif',
+ '.png' => 'image/png',
+ '.bmp' => 'image/bmp',
+ '.bitmap' => 'image/x-ms-bmp',
+ '.js' => 'application/javascript',
+ '.txt' => 'text/plain',
+ '.css' => 'text/css',
+ '.html' => 'text/html',
+ '.htm' => 'text/html'
+ }
+ end
+end
View
16 lib/mime_types.rb
@@ -1,16 +0,0 @@
-module Mime
- TYPES = {
- '.jpg' => 'image/jpg',
- '.jpeg' => 'image/jpeg',
- '.gif' => 'image/gif',
- '.png' => 'image/png',
- '.bmp' => 'image/bmp',
- '.bitmap' => 'image/x-ms-bmp',
- '.js' => 'application/javascript',
- '.txt' => 'text/plain',
- '.css' => 'text/css',
- '.html' => 'text/html',
- '.htm' => 'text/html'
- }
-
-end
View
7 lib/string_ext.rb
@@ -1,7 +0,0 @@
-
- class String #:nodoc:
- def blank?
- self !~ /\S/
- end
- end
-
View
10 script/console
@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+# File: script/console
+irb = RUBY_PLATFORM =~ /(:?mswin|mingw)/ ? 'irb.bat' : 'irb'
+
+libs = " -r irb/completion"
+# Perhaps use a console_lib to store any extra methods I may want available in the cosole
+# libs << " -r #{File.dirname(__FILE__) + '/../lib/console_lib/console_logger.rb'}"
+libs << " -r #{File.dirname(__FILE__) + '/../lib/clarity.rb'}"
+puts "Loading clarity gem"
+exec "#{irb} #{libs} --simple-prompt"
View
14 script/destroy
@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+APP_ROOT = File.expand_path(File.join(File.dirname(__FILE__), '..'))
+
+begin
+ require 'rubigen'
+rescue LoadError
+ require 'rubygems'
+ require 'rubigen'
+end
+require 'rubigen/scripts/destroy'
+
+ARGV.shift if ['--help', '-h'].include?(ARGV[0])
+RubiGen::Base.use_component_sources! [:rubygems, :newgem, :newgem_theme, :test_unit]
+RubiGen::Scripts::Destroy.new.run(ARGV)
View
14 script/generate
@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+APP_ROOT = File.expand_path(File.join(File.dirname(__FILE__), '..'))
+
+begin
+ require 'rubigen'
+rescue LoadError
+ require 'rubygems'
+ require 'rubigen'
+end
+require 'rubigen/scripts/generate'
+
+ARGV.shift if ['--help', '-h'].include?(ARGV[0])
+RubiGen::Base.use_component_sources! [:rubygems, :newgem, :newgem_theme, :test_unit]
+RubiGen::Scripts::Generate.new.run(ARGV)
View
300 server.rb
@@ -1,300 +0,0 @@
-$:.unshift File.dirname(__FILE__)
-
-require 'rubygems'
-require 'eventmachine'
-require 'evma_httpserver'
-require 'erb'
-require 'cgi'
-require 'yaml'
-require 'base64'
-require 'optparse'
-require 'lib/basic_auth'
-require 'lib/mime_types'
-require 'lib/string_ext'
-require 'lib/commands/command_builder'
-require 'lib/commands/tail_command_builder'
-require 'lib/parsers/time_parser'
-require 'lib/parsers/hostname_parser'
-require 'lib/parsers/shop_parser'
-require 'lib/renderers/log_renderer'
-
-$options = {
- :username => nil,
- :password => nil,
- :log_files => ['**/*.log*'],
- :port => 8080,
- :address => "0.0.0.0"
-}
-
-ARGV.options do |opts|
- opts.banner = "Usage: #{File.basename($PROGRAM_NAME)} [options] [directory]"
-
- opts.separator " "
- opts.separator "Specific options:"
-
- opts.on( "-f", "--config [file]", String, "Config file (yml)" ) do |opt|
- $options.update YAML.load_file( opt )
- end
-
- opts.on( "-p", "--port [port]", Integer, "Port to listen on" ) do |opt|
- $options[:port] = opt
- end
-
- opts.on( "-b", "--address [address]", String, "Address to bind to (default 0.0.0.0)" ) do |opt|
- $options[:address] = opt
- end
-
- opts.on( "--include [mask]", String, "File mask of logs to add (default: **/*.log*)" ) do |opt|
- $options[:log_files] ||= []
- $options[:log_files] += opt
- end
-
- opts.separator " "
- opts.separator "Password protection:"
-
- opts.on( "--username [USER]", String, "Optional username (httpauth)." ) do |opt|
- $options[:username] = opt
- end
-
- opts.on( "--password [PASS]", String, "Optional password (httpauth)." ) do |opt|
- $options[:password] = opt
- end
-
- opts.separator " "
- opts.separator "Misc:"
-
- opts.on( "-h", "--help", "Show this message." ) do
- puts opts
- exit
- end
-
- opts.separator " "
-
- begin
- opts.parse!
-
- if ARGV[1]
-
- end
-
- rescue
- puts opts
- exit
- end
-end
-
-module GrepRenderer
- attr_accessor :response, :parser, :marker, :params
-
- def parser
- @parser ||= TimeParser.new( HostnameParser.new(ShopParser.new), params)
- end
-
- def renderer
- @renderer ||= LogRenderer.new
- end
-
- # once download is complete, send it to client
- def receive_data(data)
- @buffer ||= StringScanner.new("")
- @buffer << data
-
- html = ""
- while line = @buffer.scan_until(/\n/)
- tokens = parser.parse(line)
- html << renderer.render(tokens)
- end
-
- return if html.empty?
-
- response.chunk html
- response.send_chunks
- end
-
- def unbind
- response.chunk '</div><hr><p id="done">Done</p></body></html>'
- response.chunk ''
- response.send_chunks
- puts 'Done'
- end
-end
-
-
-
-class Handler < EventMachine::Connection
- include EventMachine::HttpServer
- include BasicAuth
-
- LeadIn = ' ' * 1024
-
- def process_http_request
- authenticate!
-
- puts "action: #{action}"
- puts "params: #{params.inspect}"
-
- case action
- when '/'
- respond_with(200, welcome_page)
-
- when '/perform'
- if params.empty?
- respond_with(200, welcome_page)
- else
- # get command
- command = case params['tool']
- when 'grep' then CommandBuilder.new(params).command
- when 'tail' then TailCommandBuilder.new(params).command
- else raise InvalidParameterError, "Invalid Tool parameter"
- end
- response = init_chunk_response
- response.chunk results_page # display page header
-
- puts "Running: #{command}"
- EventMachine::popen(command, GrepRenderer) do |grepper|
- @grepper = grepper
- @grepper.marker = 0
- @grepper.params = params
- @grepper.response = response
- end
- end
-
- when '/test'
- response = init_chunk_response
- EventMachine::add_periodic_timer(1) do
- response.chunk "Lorem ipsum dolor sit amet<br/>"
- response.send_chunks
- end
-
- else
- requested_file = File.join(File.dirname(__FILE__), "public", ENV["PATH_INFO"])
-
- if File.exists?(requested_file)
- respond_with(200, File.open(requested_file).read, :content_type => Mime::TYPES[File.extname(requested_file)])
- else
- raise NotFoundError
- end
- end
-
- rescue InvalidParameterError => e
- respond_with(500, error_page(e))
- rescue NotFoundError => e
- respond_with(404, "<h1>Not Found</h1>")
- rescue NotAuthenticatedError => e
- puts "Could not authenticate user"
- headers = { "WWW-Authenticate" => %(Basic realm="Clarity")}
- respond_with(401, "HTTP Basic: Access denied.\n", :content_type => 'text/plain', :headers => headers)
- end
-
- def init_chunk_response
- response = EventMachine::DelegatedHttpResponse.new( self )
- response.status = 200
- response.headers['Content-Type'] = 'text/html'
- response.chunk LeadIn
- response
- end
-
- def respond_with(status, content, options = {})
- response = EventMachine::DelegatedHttpResponse.new( self )
- response.headers['Content-Type'] = options.fetch(:content_type, 'text/html')
- headers = options.fetch(:headers, {})
- headers.each_pair {|h, v| response.headers[h] = v }
- response.status = status
- p headers
- response.content = content
- response.send_response
- end
-
- def unbind
- return unless @grepper
- kill_processes(@grepper.get_status.pid)
- close_connection
- end
-
- def kill_processes(ppid)
- return if ppid.nil?
- all_pids = [ppid] + get_child_pids(ppid).flatten.uniq.compact
- puts "=== pids are #{all_pids.inspect}"
- all_pids.each do |pid|
- Process.kill('TERM',pid.to_i)
- puts "=== killing #{pid}"
- end
- rescue Exception => e
- puts "!Error killing processes: #{e}"
- end
-
- def get_child_pids(ppid)
- out = `ps -opid,ppid | grep #{ppid.to_s}`
- ids = out.split("\n").map {|line| $1 if line =~ /^\s*([0-9]+)\s.*/ }.compact
- ids.delete(ppid.to_s)
- if ids.empty?
- ids
- else
- ids << ids.map {|id| get_child_pids(id) }
- end
- end
-
- private
-
- def error_page(error)
- @error = error
- render "error.html.erb"
- end
-
- def welcome_page
- render "index.html.erb"
- end
-
- def results_page
- render "index.html.erb"
- end
-
- def render(view)
- @toolbar = template("_toolbar.html.erb")
- @content_for_header = template("_header.html.erb")
- template(view)
- end
-
- def template(filename)
- content = File.read( File.join(File.dirname(__FILE__), 'views', filename) )
- ERB.new(content).result(binding)
- end
-
- def logfiles
- $options[:log_files].map {|f| Dir[f] }.flatten.compact.uniq.select{|f| File.file?(f) }.sort
- end
-
- def authenticate!
- login, pass = authentication_data
-
- p authentication_data
-
- if ($options[:username] && $options[:username] != login) || ($options[:password] && $options[:password] != pass)
- raise NotAuthenticatedError
- end
-
- true
- end
-
- def params
- @params ||= ENV['QUERY_STRING'].split('&').inject({}) {|p,s| k,v = s.split('=');p[k.to_s] = CGI.unescape(v.to_s);p}
- end
-
- def action
- @action ||= ENV["PATH_INFO"]
- end
-end
-
-class InvalidParameterError < StandardError; end
-class NotFoundError < StandardError; end
-class NotAuthenticatedError < StandardError; end
-
-
-EventMachine::run {
- EventMachine.epoll
- EventMachine::start_server($options[:address], $options[:port], Handler)
- puts "Listening #{$options[:address]}:#{$options[:port]}..."
- puts "Adding log files: #{$options[:log_files].inspect}"
-}
-
-
View
3 test/test_helper.rb
@@ -0,0 +1,3 @@
+require 'stringio'
+require 'test/unit'
+require File.dirname(__FILE__) + '/../lib/clarity'
View
2 views/_toolbar.html.erb
@@ -83,7 +83,7 @@
<script>
Search.init({ 'grep': <%= logfiles.map {|f| f }.to_json %>,
'tail': <%= logfiles.map {|f| f if f =~ /log$/ }.compact.to_json %> },
- <%= @params.empty? ? nil.to_json : @params.to_json %> );
+ <%= params.empty? ? nil.to_json : params.to_json %> );
</script>

0 comments on commit 17b3f58

Please sign in to comment.
Something went wrong with that request. Please try again.