Skip to content

arachni/lib/arachni/page.rb:188: [BUG] Segmentation fault #305

Closed
skout23 opened this Issue Nov 9, 2012 · 4 comments

2 participants

@skout23
skout23 commented Nov 9, 2012

did a fresh install last night, under a new ruby setup on my desktop, started up the webui, added a dispatcher, lowered the concurrency from 20 to 5, changed the useragent, and setup a scan of a target that tends to 302/4 redirect all bad requests to the home page, scan progresses and I see mem and % results from the dispatcher, leave for some coffee, and at around 25-30% through the scan I get the following Seg Fault. This was the first app I thought to use with the new rbenv setup. the scanner IP is whitelisted in the WAF, but looks to be that I still hit one of our velocity thresholds, I see a few 500's from the server instead of 30x redirects, which tells me the WAF stepped in a few times (249 WAF denials out of the ~20k requests made). I don't think the WAF denials are the culprit, but just doing a brain dump to get a better picture.

Starry-NIght:~ jstout$ uname -a
Darwin Starry-NIght.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

Fresh install of rbenv
Fresh install of ruby
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-darwin11.4.2]

Fresh bundled install with local .bundle
bundle install
Using rake (0.9.2.2)
Using addressable (2.2.8)
Using arachni-rpc (0.1.2)
Using eventmachine (1.0.0)
Using em-synchrony (1.0.2)
Using arachni-rpc-em (0.1.2)
Using rack (1.4.1)
Using rack-protection (1.2.0)
Using tilt (1.3.3)
Using sinatra (1.3.3)
Using async_sinatra (1.0.0)
Using awesome_print (1.1.0)
Using bundler (1.2.1)
Using data_objects (0.10.10)
Using dm-core (1.2.0)
Using dm-aggregates (1.2.0)
Using dm-constraints (1.2.0)
Using dm-migrations (1.2.0)
Using fastercsv (1.5.5)
Using json (1.7.5)
Using json_pure (1.7.5)
Using multi_json (1.3.7)
Using dm-serializer (1.2.2)
Using dm-timestamps (1.2.0)
Using dm-transactions (1.2.0)
Using bcrypt-ruby (3.0.1)
Using stringex (1.4.0)
Using uuidtools (2.1.3)
Using dm-types (1.2.2)
Using dm-validations (1.2.0)
Using datamapper (1.2.0)
Using dm-do-adapter (1.2.0)
Using do_sqlite3 (0.10.10)
Using dm-sqlite-adapter (1.2.0)
Using net-ssh (2.6.1)
Using net-scp (1.0.4)
Using nokogiri (1.5.5)
Using i18n (0.6.1)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using pony (1.4)
Using rb-readline (0.4.2)
Using backports (2.6.5)
Using rack-test (0.6.2)
Using sinatra-contrib (1.3.2)
Using sinatra-flash (0.3.0)
Using sys-proctable (0.9.2)
Using terminal-table (1.4.5)
Using daemons (1.1.9)
Using thin (1.5.0)
Using typhoeus (0.3.3)
Using arachni (0.4.1.2) from source at .
Using diff-lcs (1.1.3)
Using rspec-core (2.11.1)
Using rspec-expectations (2.11.3)
Using rspec-mocks (2.11.3)
Using rspec (2.11.0)
Your bundle is complete! It was installed into ./.bundle

And finally the actual segfault info:

Starry-NIght:arachni jstout$ arachni_web_autostart
[>] Starting the Arachni Dispatch server...
[>] Starting the Arachni WebUI server...
[>] The web interface is at: http://127.0.0.1:4567
[>] --- It may take a while to startup, try refreshing the page a couple of times.

[>] Hit Ctrl+C to shut everything down.
[-] Failed to parse 'https://app_contacts.html'.
[-] Failed to parse 'https://app_contacts.html'.
[-] Failed to parse 'https://app_faqs.html'.
[-] Failed to parse 'https://app_faqs.html'.
[-] Failed to parse 'https://our_app.html'.
[-] Failed to parse 'https://our_app.html'.
[-] Failed to parse 'https://app_faqs.html#1'.
[-] Failed to parse 'https://app_faqs.html#1'.
[-] Failed to parse 'https://app_faqs.html#2'.
[-] Failed to parse 'https://app_faqs.html#2'.
[-] Failed to parse 'https://app_faqs.html#3'.
[-] Failed to parse 'https://app_faqs.html#3'.
[-] Failed to parse 'https://app_faqs.html#4'.
[-] Failed to parse 'https://app_faqs.html#4'.
[-] Failed to parse 'https://app_faqs.html#5'.
[-] Failed to parse 'https://app_faqs.html#5'.
[-] Failed to parse 'https://app_faqs.html#6'.
[-] Failed to parse 'https://app_faqs.html#6'.
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:188: [BUG] Segmentation fault
ruby 1.9.3p286 (2012-10-12 revision 37165) [x86_64-darwin11.4.2]

-- Control frame information -----------------------------------------------
c:0043 p:---- s:0150 b:0150 l:000149 d:000149 CFUNC :dup
c:0042 p:0012 s:0147 b:0147 l:000146 d:000146 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:188
c:0041 p:0027 s:0143 b:0141 l:000131 d:000140 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:124
c:0040 p:---- s:0137 b:0137 l:000136 d:000136 FINISH
c:0039 p:---- s:0135 b:0135 l:000134 d:000134 CFUNC :each
c:0038 p:0016 s:0132 b:0132 l:000131 d:000131 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:124
c:0037 p:---- s:0128 b:0128 l:000127 d:000127 FINISH
c:0036 p:---- s:0126 b:0126 l:000125 d:000125 CFUNC :new
c:0035 p:0508 s:0122 b:0122 l:000121 d:000121 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/parser.rb:212
c:0034 p:0036 s:0112 b:0112 l:000111 d:000111 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:119
c:0033 p:0059 s:0107 b:0106 l:0023f8 d:000105 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:108
c:0032 p:---- s:0103 b:0103 l:000102 d:000102 FINISH
c:0031 p:---- s:0101 b:0101 l:000100 d:000100 CFUNC :call
c:0030 p:0021 s:0097 b:0096 l:000087 d:000095 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/typhoeus/request.rb:54
c:0029 p:---- s:0093 b:0093 l:000092 d:000092 FINISH
c:0028 p:---- s:0091 b:0091 l:000090 d:000090 CFUNC :each
c:0027 p:0033 s:0088 b:0088 l:000087 d:000087 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/typhoeus/request.rb:53
c:0026 p:0103 s:0085 b:0085 l:000084 d:000084 METHOD /Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:222
c:0025 p:0035 s:0078 b:0078 l:0014a8 d:000077 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:186
c:0024 p:---- s:0074 b:0074 l:000073 d:000073 FINISH
c:0023 p:---- s:0072 b:0072 l:000071 d:000071 CFUNC :call
c:0022 p:0021 s:0068 b:0068 l:000067 d:000067 METHOD /Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/easy.rb:362
c:0021 p:---- s:0065 b:0065 l:000064 d:000064 FINISH
c:0020 p:---- s:0063 b:0063 l:000062 d:000062 CFUNC :multi_perform
c:0019 p:0019 s:0060 b:0060 l:000059 d:000059 METHOD /Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/multi.rb:21
c:0018 p:0131 s:0057 b:0057 l:000056 d:000056 METHOD /Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:95
c:0017 p:0061 s:0051 b:0051 l:000050 d:000050 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/http.rb:597
c:0016 p:0015 s:0048 b:0048 l:001c48 d:000047 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/http.rb:187
c:0015 p:---- s:0046 b:0046 l:000045 d:000045 FINISH
c:0014 p:---- s:0044 b:0044 l:000043 d:000043 CFUNC :call
c:0013 p:0016 s:0041 b:0041 l:000040 d:000040 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/utilities.rb:276
c:0012 p:0011 s:0035 b:0035 l:001c48 d:001c48 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/http.rb:185
c:0011 p:0073 s:0032 b:0032 l:000031 d:000031 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:762
c:0010 p:0073 s:0029 b:0029 l:000158 d:000158 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:667
c:0009 p:0152 s:0026 b:0026 l:002398 d:002398 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:633
c:0008 p:0009 s:0023 b:0023 l:0012d0 d:000022 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:188
c:0007 p:---- s:0021 b:0021 l:000020 d:000020 FINISH
c:0006 p:---- s:0019 b:0019 l:000018 d:000018 CFUNC :call
c:0005 p:0016 s:0016 b:0016 l:000015 d:000015 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/utilities.rb:276
c:0004 p:0023 s:0010 b:0010 l:0012d0 d:0012d0 METHOD /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:188
c:0003 p:0008 s:0006 b:0006 l:0014d8 d:000005 BLOCK /Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/rpc/server/framework.rb:268
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:---- s:0002 b:0002 l:000001 d:000001 TOP

-- Ruby level backtrace information ----------------------------------------
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/rpc/server/framework.rb:268:in block in run'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:188:in
run'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/utilities.rb:276:in exception_jail'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/utilities.rb:276:in
call'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:188:in block in run'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:633:in
audit'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:667:in audit_queue'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/framework.rb:762:in
harvest_http_responses'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/http.rb:185:in run'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/utilities.rb:276:in
exception_jail'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/utilities.rb:276:in call'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/http.rb:187:in
block in run'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/http.rb:597:in hydra_run'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:95:in
run'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/multi.rb:21:in perform'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/multi.rb:21:in
multi_perform'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/easy.rb:362:in success'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/easy.rb:362:in
call'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:186:in block in get_easy_object'
/Users/jstout/Media/Projects/Ruby/arachni/.bundle/ruby/1.9.1/gems/typhoeus-0.3.3/lib/typhoeus/hydra.rb:222:in
handle_request'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/typhoeus/request.rb:53:in call_handlers'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/typhoeus/request.rb:53:in
each'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/typhoeus/request.rb:54:in block in call_handlers'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/typhoeus/request.rb:54:in
call'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:108:in block (2 levels) in from_url'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:119:in
from_response'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/parser.rb:212:in page'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/parser.rb:212:in
new'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:124:in initialize'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:124:in
each'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:124:in block in initialize'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:188:in
try_dup'
/Users/jstout/Media/Projects/Ruby/arachni/lib/arachni/page.rb:188:in `dup'

Cheers,
Scott

@Zapotek
Arachni - Web Application Security Scanner Framework member
Zapotek commented Nov 9, 2012

The backtrace it a bit misleading, the issue is somewhere in Typhoeus.
The problem is that I haven't yet managed to find a way to reliably reproduce this but it looks like it only happens when Threads are involved.
I've added some synchronization code to the HTTP lib in the "feature/distributed-crawling" branch I've been working on recently and it seems to have taken care of the problem so I'll port these changes to the experimental branch as well and let you know so that you can use it.

@skout23
skout23 commented Nov 9, 2012

Swank, I played around with Roundabout about 2 months back, fun stuff. Is that where your heading? Either way I will switch branches and give it a spin.

@Zapotek
Arachni - Web Application Security Scanner Framework member
Zapotek commented Nov 9, 2012

Yeah I've implemented that in Arachni and I'm now testing and bugfixing. The branch is considered unstable though.

@Zapotek
Arachni - Web Application Security Scanner Framework member
Zapotek commented Mar 30, 2013

Newer versions of Ruby seem to have fixed this problem, closing.

@Zapotek Zapotek closed this Mar 30, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.