Host-Extract | Host/IP Pattern Extractor
========================================

This little ruby script tries to extract all IP/Host patterns in page response of a given URL and JavaScript/CSS files of that URL.

With it, you can quickly identify internal IPs/Hostnames, development IPs/ports, cdn, load balancers, additional attack entries related to your target that are revealed in inline js, css, html comment areas and js/css files.

This is unlike web crawler which looks for new links only in anchor tags (<a) or the like. [you might miss many additional targets if you ever use such web crawler or other GUI-based tools that shows you your main target and its relationship with its linked sub/off-site domains]

In some cases, host-extract may give you false positives when there are some words like - main-site_ver_10.2.1.3.swf.

With -v option, you can ask the tool to output html view-source snippets for each IP/Domain extracted. This will shorten your manual analysis time.



Download/Update
================

svn co http://host-extract.googlecode.com/svn/trunk/ host-extract


USAGE 
======

Usage: host-extract.rb [options]
    -a                   find all ip/host patterns
    -j                   scan all js files
    -c                   scan all css files
    -v                   append view-source html snippet for manual verification


A sugguested usage is to feed lists of urls output by your favorite web crawler.

In Win OSes, you can

for  /F "tokens=*" %G IN ('type url-list') do ruby host-extract.rb -a -v %G
In Linux platforms, you can

for line in $(cat url-list); do ruby host-extract.rb -a -v $line; done

The run.cmd and run.sh are included to be ready for use.

Windows: run url-list

Linux: sh run.sh url-list


Tutorial Wiki
==============

Thanks to Sebastien Damaye from aldeid.com, you can see a thorough host-extract tutorial with real-world web sites that reveal various juicy info shown inside html/js sources. 

http://aldeid.com/index.php/Host-extract


Bugs/Suggestions Report
========================

Please report Bugs/Suggestions to host-extract at yehg.net. 
Thanks for your contribution.