checking in all the files

README

@@ -0,0 +1,117 @@
+Pipal, Password Analyser
+========================
+
+Copyright(c) 2012, Robin Wood <robin@digininja.org>
+
+On most internal pen-tests I do I generally manage to get a password dump from
+the DC. To do some basic analysis on this I wrote Counter and since I originally
+released it I've made quite a few mods to it to generate extra stats that are
+useful when doing reports to management. 
+
+Recently a good friend, n00bz, asked on Twitter if anyone had a tool that he
+could use to analyse some passwords he had. I pointed him to Counter and said if
+he had any suggestions for additions to let me know. He did just that and over
+the last month between us we have come up with a load of new features which we
+both think will help anyone with a large dump of cracked passwords to analyse.
+We also got some input from well known password analysts Matt Weir and Martin
+Bos who I'd like to give a big thanks to. 
+
+I have to point out before going on, all this tool does is to give you the stats
+and the information to help you analyse the passwords. The real work is done by
+you in interpreting the results, I give you the numbers, you tell the story. 
+
+Seeing as there have been so many changes to the underlying code I also decided
+to change the name (see below) and do a full new release.
+
+So, what does this new version do? The best way to describe it is to see some
+examples so go to the Pipal project page at www.digininja.org/projects/pipal.php
+for a full walk through of a sample analysis.
+
+Install / Usage
+===============
+The app will only work with Ruby 1.9.x, if you try to run it in any previous
+versions you will get a warning and the app will close.
+
+Pipal is completely self contained and requires no gems installing so should
+work on any vanilla Ruby install.
+
+Usage is fairly simple, -? will give you full instructions:
+
+$ ./pipal.rb -?
+pipal 1.0 Robin Wood (robin@digininja.org) (www.digininja.org)
+
+Usage: pipal [OPTION] ... FILENAME
+        --help, -h: show help
+        --top, -t X: show the top X results (default 10)
+        --output, -o : output to file
+        --external, -e : external file to compare words against
+
+        FILENAME: The file to count 
+
+When you run the app you'll get a nice progress bar which gives you a rough idea
+of how long the app will take to run. If you want to stop it at any point
+hitting ctrl-c will stop the parsing and will dump out the stats generated so
+far.
+
+The progress bar is based on a line count from the file which it gets this using
+the wc command. If it can't find wc it will make a guess at the number of lines
+based on the file size and an average line length of 8 bytes so the progress bar
+may not be fully accurate but should still give you an idea.
+
+Version History
+===============
+
+Version 2 - Two big changes, the first a massive speed increase. This patch was
+submitted by Stefan Venken who said a small mention would be good enough, I want
+to give him a big mention. Running through the LinkedIn lists would have taken
+many many hours on version 1, version 2 went through 3.5 million records in
+about 15 minutes. Thank you.
+
+Second change is the addition of US area and zip code lookups. This little
+feature gives some interesting geographical data when ran across password lists
+originating in the US. The best example I've seen of this is the dump from the
+Military Singles site where some passwords could be obviously seen to be grouped
+around US military bases. People in the UK don't have the same relationship with
+phone numbers so I know this won't work here but if anyone can suggest any other
+areas where this might be useful then I'll look at building in some kind of
+location awareness feature so you can specify the source of the list and get
+results customized to the correct area or just run every area and see if a
+pattern emerges.
+
+A non-code-base change is for version 2 is the move from hosting the code myself
+to github. This is my first github hosted project so I may get things wrong, if
+I do, sorry. A number of people asked how they could submit patches so this
+seems like the best way to do it, lets hope it works out.
+
+Version 1 - Was a proof of concept, written fairly in a fairly verbose way so not
+very optimised. Took off way more than I expected it would and gathered a lot of
+community support.
+
+Feedback/Todo
+=============
+
+If you have a read through the source for Pipal you'll notice that it isn't very
+efficient at the moment. The way I built it was to try to keep each chunk of
+stats together as a distinct group so that if I wanted to add a new, similar,
+group then it was easy to just copy and paste the group. Now I've got a working
+app and I know roughly what I need in the different group types I've got an idea
+on how to rewrite the main parser to make it much more efficient and hopefully
+multi-threaded which should speed up the processing by a lot for large lists. 
+
+I could have made these changes before releasing version 1.0 but I figured
+before I do I want to get as much feedback as possible from users about the
+features already implemented and about any new features they would like to see
+so that I can bundle all these together into version 2. So, please get in touch
+if there is a set of stats that you'd like to see included. 
+
+One other thing I know needs fixing, Pipal doesn't handle certain character
+encodings very well. If anyone knows how to correctly deal with different
+encoding types, especially with regards to regular expressions, please let me
+know.
+
+Licence
+=======
+This project released under the Creative Commons Attribution-Share Alike 2.0
+UK: England & Wales
+
+( http://creativecommons.org/licenses/by-sa/2.0/uk/ )

horizbar.rb

@@ -0,0 +1,53 @@
+# This is a slightly modified version of the HorizBar script taken from
+# https://blogs.oracle.com/realneel/entry/ascii_graphs_using_ruby
+
+class HorizBar
+	WIDTH = 72
+	HEIGHT = 16
+	attr :output_file, true
+
+	def initialize(array)
+		@values = array
+		@output_file = STDOUT
+	end
+
+	def draw
+		#Adjust X axis when there are more than WIDTH cols
+		if @values.length > WIDTH then
+			old_values = @values;
+			@values = []
+			0.upto(WIDTH - 1){ |i| @values << old_values[i*old_values.length/WIDTH]}
+		end
+
+		max = 0
+		@values.each do |val|
+			if !val.nil? and max < val
+				max = val
+			end
+		end
+		# can't use this as the array can have nil's in it
+		# and max can't cope with that
+		# max = @values.max
+		if max == 0
+			return
+		end
+
+		# initialize display with blanks
+		display = Array.new(HEIGHT).collect { Array.new(WIDTH, ' ') }
+		@values.each_with_index do |e, i|
+			f = (e.nil?)?0:e
+
+			num= f*HEIGHT/max
+			(HEIGHT - 1).downto(HEIGHT - 1 - num){|j| display[j][i] = '|'}
+		end    
+		display.each{|ar| ar.each{|e| @output_file.putc e}; @output_file.puts "\n"} #now print
+
+		no_of_digits = (@values.length - 1).to_s.length
+		0.upto(no_of_digits) do |digit_number|
+			0.upto(@values.length - 1) do |x|
+				@output_file.print sprintf("%0#{no_of_digits}d", x)[digit_number]
+			end
+			@output_file.puts
+		end
+	end
+end