Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
An easy-to-use and extensible framework for OSINT.
Ruby JavaScript CSS Other

README.md

TAPIR - The API for Reconaissance

Support

The project is currently on hold. For inquries, contact jcran@intrigue.io.

About

TAPIR is a framework designed to make it easy to discover data about entities - organizations, users, computers, and networks - on the web, using common (and not so common) OSINT techniques.

At the core of TAPIR are entities, implemented and database-backed with MongoDB, and the tasks (implemented as small, structured ruby scripts like Metasploit modules) to modify and create entities. TAPIR entities are real-world objects that you want to discover more information about.

TAPIR tasks are the code that operate on the entities to create findings, new entities, or modify existing entities. Tasks are simple to create, have just enough structure, and harness the power of Ruby to extend the framework in useful ways. Have a look at the existing tasks in the lib/tapir/tasks directory.

TAPIR keeps track of entities generated by each task for you. For example, if you add a host entity, and run a 'geolocate_host' task, you'll find that the physical address generated by the task is now a child of that host (and the host is now a parent of that physical address). You can view, modify, and programmatically query and inspect these relationships.

News

  • 08/15/2014 - Rework for scale. Projects now scale to 40,000+ entities. 45 tasks.
  • 07/16/2014 - Project brought back online, total of 42 tasks.
  • 01/01/2014 - Project on hold.
  • 08/22/2013 - TAPIR now supports multiple projects, and we have a total of 34 tasks!
  • 05/07/2013 - Mongo is now used as the datastore. Generic Entities!
  • 06/18/2012 - Cleaned up Web UI and background tasks! Renamed to TAPIR!
  • 02/25/2012 - The EAR Project has a stubbed out web UI, and is on its way to v1.0!
  • 12/16/2011 - The EAR Project has been updated to Rails 3!
  • 06/01/2011 - (or some time around here) Initial version of EAR spawned for #AHA.

Prerequisites:

TAPIR is currently tested and working on:

  • OS X 10.8.x+
  • Ubuntu Linux 9.10+
  • Kali Linux 1.0

Prerequisites Installation (OSX using Homebrew)

Brew can be used to install prerequisites on OSX:

brew install qt
brew install nmap
brew install mongo

You'll want to start mongo by executing the following:

$ sudo mongod

Prerequisites Installation (Tested on Ubuntu 14.04)

These prerequisites can be installed via apt:

sudo apt-get install nmap libxslt-dev libxml2-dev libpcap-dev 

Mongo can be installed with the following:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list # if you're on ubuntu
sudo apt-get update
sudo apt-get install mongodb-10gen

See: http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/ for more information

Masscan can be installed with the following:

sudo apt-get install git gcc make libpcap-dev
git clone https://github.com/robertdavidgraham/masscan
cd masscan
make
sudo make install

Prerequisites Installation (Kali Linux 1.0+)

These prerequisites can be installed via apt:

sudo apt-get install nmap qt4-qmake libnokogiri-ruby1.8 libxslt-dev libxml2-dev libqt4-dev libpcap-dev libpq-dev libsqlite3-dev 

Mongo can be installed with the following:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list
sudo apt-get update
sudo apt-get install mongodb-10gen

Initial TAPIR setup (platform independent):

You'll want to have RVM installed and configured with Ruby 1.9.3. To do this, simply use the installer script for RVM and once completed, execute the following commands

$ rvm install 1.9.3
$ rvm use 1.9.3 --default

Navigate to the root of the TAPIR directory and install the ruby dependencies:

$ cd $TAPIR_ROOT
$ gem install bundle                
$ bundle install                    
$ bundle exec rake secret > .secret 

Getting Started with TAPIR

Using the Web Interface (After you've installed the prerequisites!)

To start the server, in the root of the TAPIR directory, run:

$ RAILS_ENV=development bundle exec rackup                # to start the server

Now browse to http://[server_name]:9292

In order to log in, you'll need to generate a username and password - Make sure you do this AFTER browsing to the web interface for the first time.

$ RAILS_ENV=development bundle exec rake db:seed           # to generate a username and password

Configure data sources

Most data sources are available with just an internet connection. Some require you to pull down data in advance.

The latest geolitecity (geolocation) data can be pulled by running:

$ cd $TAPIR_ROOT/data
$ ./geolitecity/get_latest.sh 

Using the Scriptable Console (Advanced)

Once you have a database, simply run $ bundle exec ./util/console.rb - this will give you access to a shell from which you can create entities and run tasks against them.

Creating a host entity & running tasks:

$ cd $TAPIR_ROOT/util
$ bundle exec ./console.rb

    TAPIR> Tenant.current = Tenant.first
    TAPIR> Project.current = Project.first
    TAPIR> host = Entities::Host.create(:name => "8.8.8.8")
    TAPIR> host.run_task("dns_reverse_lookup",{})
    TAPIR> host.run_task("geolocate_host",{})
    TAPIR> host.children.map{ |c| puts "#{c.class}: #{c.name}" };

Production

In production, resque is used for background tasks. Background workers can be enabled with the following:

$ cd $TAPIR_ROOT

$ bundle install && bundle exec rake assets:clean && bundle exec rake assets:precompile
$ bundle exec god terminate && bundle exec god -c god_config
$ RAILS_ENV=production bundle exec rackup
Something went wrong with that request. Please try again.