Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100755 141 lines (117 sloc) 3.438 kb
884a29d0 »
2009-07-31 - More descriptive title for the Quantcast list
1 #!/usr/bin/env ruby
b92b7810 »
2009-07-23 Got ip-to-asn working!
2 # encoding: UTF-8
3 $LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/lib')
4
5 require 'pp'
06785b7f »
2009-07-25 Rename all instances of domain-profile to domain-profiler
6 require 'domain-profiler'
b92b7810 »
2009-07-23 Got ip-to-asn working!
7 require 'rubygems'
8 require 'gchart'
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
9 require 'erb'
b92b7810 »
2009-07-23 Got ip-to-asn working!
10
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
11 def fullname(name)
12 Name.new.full(name)
13 end
14
b92b7810 »
2009-07-23 Got ip-to-asn working!
15 class Array
16 def lookup(host)
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
17 self.map{|name| Name.new.simplify(name,host) }.uniq
b92b7810 »
2009-07-23 Got ip-to-asn working!
18 end
19 end
20
21
22e0704f »
2011-01-06 Add usage info to profile-list
22 if ARGV.length == 0
23 print "Usage: ./profile-list [file containing one domain per line] 'header to put in HTML output'\n"
24 exit
25 end
26
b92b7810 »
2009-07-23 Got ip-to-asn working!
27 filename = ARGV[0]
3a2425ae »
2009-07-23 Update profile to make an array of DomainProfile classes. Next step: …
28 file = File.new(filename)
b92b7810 »
2009-07-23 Got ip-to-asn working!
29
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
30 hosts = {}
1699fde7 »
2011-01-06 Add second argument to profile-list to define header contents in gene…
31 list_name = ARGV[1] ? ARGV[1] : 'REPLACE ME'
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
32
33 file.map {|host|
dfc375db »
2009-07-29 - Styling that I'm finally happy with.
34 if match_data = host.match(/^#List:(.*)/)
35 list_name = match_data[1]
36 end
33fc439d »
2009-07-27 Update the output from the profile command to look better
37 next if host.match(/^#/)
b92b7810 »
2009-07-23 Got ip-to-asn working!
38 host.chomp!
06785b7f »
2009-07-25 Rename all instances of domain-profile to domain-profiler
39 profile = DomainProfiler.new(host)
b92b7810 »
2009-07-23 Got ip-to-asn working!
40
2ab2da3d »
2011-12-22 add try/catch block to profile-list to handle unexpected or strange e…
41 begin
42 out = {}
43 out[:web_host] = profile.dns.a.map{|record| orgname(record.answer) }.lookup(host)
44 out[:dns_host] = profile.dns.ns.map{|record| Name.new.shorten(record.answer) }.lookup(host)
45 out[:mail_host] = profile.dns.mx.map{|record| Name.new.shorten(record.host) }.lookup(host)
46 out[:registrar] = profile.whois.registrar.lookup(host)
47 out[:ssl_issuer] = profile.ssl.ca.lookup(host)
48 out[:ssl_type] = profile.ssl.cn
49 hosts[host] = out
50 rescue
51 end
52
b92b7810 »
2009-07-23 Got ip-to-asn working!
53 }
54
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
55 # Modify this to make stats on hosting provider, dns, mail, whois, ssl, ssn type and count HTML
56
57 count = {}
58 types = [:web_host,:mail_host,:dns_host,:registrar,:ssl_issuer,:ssl_type]
59
60 types.each { |kind| count[kind] = [] }
61
62 # Turn the list of host data into a hash of type data
63 hosts.each do |hostname,data|
64 data.each do |kind,value|
5741f3aa »
2009-07-27 Remove the orgname function and use the DomainProfiler one instead, i…
65 if kind === :ssl_type and not value[0].is_a? Symbol
66 if value[0].match(/^\*/)
67 ssl_type = :star
68 else
69 ssl_type = :normal
70 end
71
72 value = ssl_type
babb4c22 »
2009-07-25 Update Name class with new :netsol pattern, Change profile.rb to show…
73 end
74
5741f3aa »
2009-07-27 Remove the orgname function and use the DomainProfiler one instead, i…
75 count[kind].push(value)
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
76 end
77 end
78
79 output = {}
80 count.each do |kind,values|
81 summary = {}
82 values.flatten.each do |value|
83 value = :other unless value.is_a? Symbol
84
85 if summary[value].is_a? Integer
86 summary[value] += 1
87 else
88 summary[value] = 1
89 end
90 end
91 output[kind] = summary
92 end
93
94 # Collapse any item less than smallest_percent into the :other category
95 smallest_percent = 0.02
96 output.each do |kind,values|
97 total = 1
98 values.each {|k,v| total += v}
99 smallest_value = total * smallest_percent
100 values.each {|k,v|
101 if v < smallest_value
cd917af2 »
2009-07-31 Add "Domain Bank" to the lookup table, fix a bug in profile.rb that o…
102 values[:other] = 1 unless values[:other].is_a? Integer
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
103 values[:other] += v
104 values.delete(k)
105 end
106 }
107 end
108
33fc439d »
2009-07-27 Update the output from the profile command to look better
109 full_name = {
110 :web_host => 'Web Host',
111 :mail_host => 'Email Host',
112 :dns_host => 'DNS Host',
113 :registrar => 'Registrar',
114 :ssl_issuer => 'SSL Issuer',
115 :ssl_type => 'Certificate Type',
116 }
117
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
118
119 output.each do |kind, summary_data|
120 keys = []
121 values = []
122 summary_data.sort { |a,b| a[1] <=> b[1] }.each do |k,v|
5741f3aa »
2009-07-27 Remove the orgname function and use the DomainProfiler one instead, i…
123 keys.push(Name.new.full(k))
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
124 values.push(v)
125 end
126
33fc439d »
2009-07-27 Update the output from the profile command to look better
127 output[kind] = Gchart.pie(:size => '400x200',
128 # :background => 'F8ECDC',
129 :bar_color => ['CFF09E','A8DBA8','79BD9A','3B8686','0B486B'],
130 :color => '',
131 :title => full_name[kind],
132 :labels => keys,
133 :data => values
134 )
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
135 end
136
33fc439d »
2009-07-27 Update the output from the profile command to look better
137 charts = []
138 types.each do |kind|
139 charts.push( {:kind => kind, :url => output[kind] } )
140 end
6ef96530 »
2009-07-25 Checkpoint for profile.rb, also:
141 puts ERB.new(File.read("view/html")).result
Something went wrong with that request. Please try again.