Skip to content

Commit

Permalink
fix documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
trolldbois committed Nov 5, 2015
1 parent 221035e commit cb75ee7
Show file tree
Hide file tree
Showing 3 changed files with 701 additions and 451 deletions.
53 changes: 34 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
Quick Start:
============
[Quick usage guide](docs/Haystack basic usage.ipynb) in the docs/ folder.
[Haystack-reverse CLI](docs/Haystack reverse CLI.ipynb) in the docs/ folder.

Introduction:
=============
Expand All @@ -18,18 +19,17 @@ C structure matching.
The first function/API is the SEARCH function.
- It gives the ability to search for known record types in a process memory dump (or live process's memory)

**alpha-stage**
The second function/API is the REVERSE function.
- It aims at giving a reverse engineering look
at a memory dump, focusing on reconstruction, classification of classic
C structures from memory. Heap analysis. Dynamic types definition.
- It aims at helping an analyst in reverse engineering the memory records types present in a process heap.
It focuses on reconstruction, classification of classic C structures from memory.
It attempts to recreate types definition.

How to get a memory dump:
=========================

While technically you could use a third party tool, haystack actually
need memory mapping information to work with.
So there is a dumping tool included::
So there is a dumping tool included:

$ sudo haystack-dump dump <pid> dumps/myssh.dump

Expand All @@ -41,6 +41,7 @@ containing each memory map in a separate file :

Or you can write a `haystack.abc.IMemoryMapping` implementation for your favorite format.
There is already a beta volatility support in `haystack.mappings.vol`
And there is a volatility-to-haystack process dump in `haystack.mappings.vol2map`

Search for known structures:
============================
Expand Down Expand Up @@ -83,11 +84,17 @@ Example:
myfield: [1,0xff]
ptr_field: NotNull

Dynamic constraints definition:
-------------------------------
You can also create more complex constraints using python code by implementing
a `haystack.abc.interface.IRecordTypeDynamicConstraintsValidator` class and feeding it to
the `ModuleConstraints.set_dynamic_constraints`


Command line example:
---------------------

**sslsnoop repository needs an update to be compatible with releases > v0.20 - pending**
**sslsnoop repository needs an update to be compatible with releases > v0.30 - pending**

For example, this will dump the session_state structures + pointed
children structures as an python object that we can play with.
Expand Down Expand Up @@ -130,47 +137,55 @@ C Headers.
Or define your python ctypes record by hand.


Heap analysis / MemoryHandler Reverser / MemoryHandler forensics:
Heap analysis / forensics:
===================================================

**alpha-stage-not-working**

Quick info:
The `haystack-reverse` tool parse the heap for allocator structures, pointers
values, small integers and text (ascii/utf).
Given all the previous information, it can extract instances
and helps you in classifying and defining structures types.

IPython notebook usage guide:
[Haystack-reverse CLI](docs/Haystack reverse CLI.ipynb) in the docs/ folder.

Command line example:
--------------------
This will create several files in the folder containing <yourdumpname>:
The first step is to launch the analysis process with the `analyze` command.
This will create several files in the `cache/` folder in the memory dump folder:

$ python haystack-reverse <yourdumpfolder> instances
$ python haystack-reverse haystack/test/src/test-ctypes6.64.dump instances
$ python haystack-reverse haystack/test/src/test-ctypes6.64.dump analyze
$ ls -l haystack/test/src/test-ctypes6.64.dump/cache
$ ls -l haystack/test/src/test-ctypes6.64.dump/cache/structs

The most interesting one being the `<yourdumpfolder>/cache/headers_values.py` that
This will create a few files. The most interesting one being the `<yourdumpfolder>/cache/xxxxx.headers_values.py` that
gives you an ctypes listing of all found structures, with guesstimates
on fields types.

A `<yourdumpfolder>/cache/graph.gexf` file is also produced to help you visualize
instances links. It gets messy for any kind of serious application.

Other commands are listed below.

Show the list of heuristicly reversed record types:

$ python haystack-reverse <yourdumpname> types

Show the list of strings field in record:

Show ordered list of structures, by similarities:
$ python haystack-reverse <yourdumpname> strings

$ python haystack-reverse <yourdumpname> show
Show the record for a specific address:

Show only structures of size *324*::
$ python haystack-reverse <yourdumpname> show 0x00ab0000

$ python haystack-reverse <yourdumpname> show --size 324
Show the bytes hexadecimal values for the record for a specific address:

$ python haystack-reverse <yourdumpname> hex 0x00ab0000

Write to file an attempt to reversed the original types hierachy:
Show the record, if any, that has a pointer to the record sitting at a specific address:

$ python haystack-reverse <yourdumpname> typemap
$ python haystack-reverse <yourdumpname> parents 0x00ab0000

Clean the cache created :

Expand Down
131 changes: 71 additions & 60 deletions docs/Haystack & Volatility demo.ipynb
Original file line number Diff line number Diff line change
@@ -1,66 +1,77 @@
{
"metadata": {
"name": "Haystack & Volatility demo"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
"cells": [
{
"cells": [
{
"cell_type": "code",
"collapsed": false,
"input": "import haystack",
"language": "python",
"metadata": {},
"outputs": [],
"prompt_number": 1
},
{
"cell_type": "code",
"collapsed": false,
"input": "f = '/home/other/outputs/vol/zeus.vmem'\npid = 676 # services.exe\nfrom haystack.mappings import vol\nmapper = vol.VolatilityProcessMapper(f, pid)\nmappings = mapper.getMappings()\n\nfrom haystack import abouchet\nstructType = abouchet.getKlass(classname)\nheaps = abouchet._search(mappings, structType, fullscan=True)\n\nfor x in mappings:\n print x.start, \"Signature:\", hex(x.readStruct(x.start,winheap.HEAP).Signature)\n",
"language": "python",
"metadata": {},
"outputs": [
{
"ename": "SystemExit",
"evalue": "2",
"output_type": "pyerr",
"traceback": [
"An exception has occurred, use %tb to see the full traceback.\n",
"\u001b[0;31mSystemExit\u001b[0m\u001b[0;31m:\u001b[0m 2\n"
]
}
],
"prompt_number": 3
},
{
"cell_type": "code",
"collapsed": false,
"input": "%tb",
"language": "python",
"metadata": {},
"outputs": [
{
"ename": "AttributeError",
"evalue": "'file' object has no attribute 'set_parent'",
"output_type": "pyerr",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mAttributeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;31mAttributeError\u001b[0m: 'file' object has no attribute 'set_parent'"
]
}
],
"prompt_number": 6
},
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import haystack"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"cell_type": "raw",
"metadata": {},
"source": "Seen in ipython console:\n\n*** Failed to import volatility.plugins.ssdt (NameError: name 'distorm3' is not defined)\n*** Failed to import volatility.plugins.malware.threads (NameError: name 'distorm3' is not defined)\n*** Failed to import volatility.plugins.mac.apihooks_kernel (ImportError: No module named distorm3)\n*** Failed to import volatility.plugins.linux.netscan (ImportError: No module named yara)\n*** Failed to import volatility.plugins.malware.apihooks (NameError: name 'distorm3' is not defined)\n*** Failed to import volatility.plugins.mac.check_syscall_shadow (ImportError: No module named distorm3)\n*** Failed to import volatility.plugins.dumpcerts (NameError: name 'yara' is not defined)\n*** Failed to import volatility.plugins.mac.apihooks (ImportError: No module named distorm3)\nUsage: -c [options]\n\n-c: error: no such option: --IPKernelApp.parent_appname\n"
"ename": "SystemExit",
"evalue": "2",
"output_type": "error",
"traceback": [
"An exception has occurred, use %tb to see the full traceback.\n",
"\u001b[0;31mSystemExit\u001b[0m\u001b[0;31m:\u001b[0m 2\n"
]
}
],
"metadata": {}
"source": [
"f = '/home/other/outputs/vol/zeus.vmem'\n",
"pid = 676 # services.exe\n",
"from haystack.mappings import vol\n",
"mapper = vol.VolatilityProcessMapper(f, pid)\n",
"mappings = mapper.getMappings()\n",
"\n",
"from haystack import abouchet\n",
"structType = abouchet.getKlass(classname)\n",
"heaps = abouchet._search(mappings, structType, fullscan=True)\n",
"\n",
"for x in mappings:\n",
" print x.start, \"Signature:\", hex(x.readStruct(x.start,winheap.HEAP).Signature)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"Go see https://github.com/trolldbois/volatility_plugins"
]
}
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Loading

0 comments on commit cb75ee7

Please sign in to comment.