-
Notifications
You must be signed in to change notification settings - Fork 4
/
README
167 lines (109 loc) · 6.49 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
8 888888888o. .8. 8 888888888o. 8 8888 ,88' 8888888 8888888888 ,o888888o. ,o888888o. 8 8888 d888888o.
8 8888 `^888. .888. 8 8888 `88. 8 8888 ,88' 8 8888 . 8888 `88. . 8888 `88. 8 8888 .`8888:' `88.
8 8888 `88. :88888. 8 8888 `88 8 8888 ,88' 8 8888 ,8 8888 `8b ,8 8888 `8b 8 8888 8.`8888. Y8
8 8888 `88 . `88888. 8 8888 ,88 8 8888 ,88' 8 8888 88 8888 `8b 88 8888 `8b 8 8888 `8.`8888.
8 8888 88 .8. `88888. 8 8888. ,88' 8 8888 ,88' 8 8888 88 8888 88 88 8888 88 8 8888 `8.`8888.
8 8888 88 .8`8. `88888. 8 888888888P' 8 8888 88' 8 8888 88 8888 88 88 8888 88 8 8888 `8.`8888.
8 8888 ,88 .8' `8. `88888. 8 8888`8b 8 888888< 8 8888 88 8888 ,8P 88 8888 ,8P 8 8888 `8.`8888.
8 8888 ,88'.8' `8. `88888. 8 8888 `8b. 8 8888 `Y8. 8 8888 `8 8888 ,8P `8 8888 ,8P 8 8888 8b `8.`8888.
8 8888 ,o88P' .888888888. `88888. 8 8888 `8b. 8 8888 `Y8. 8 8888 ` 8888 ,88' ` 8888 ,88' 8 8888 `8b. ;8.`8888
8 888888888P' .8' `8. `88888. 8 8888 `88. 8 8888 `Y8. 8 8888 `8888888P' `8888888P' 8 888888888888 `Y8888P ,88P'
Alpha v0.0.1 Release!
A PENTESTERS GUIDE TO PWNAGE VISUALIZATION
I. Background
This is the alpha release of the DarkTools framework.
Designed from the ground up to visualize within Splunk output from common penetration testing tools.
It currently features:
- automating common penetration testing tasks
- normalizing the output into JSON objects
- sending the data to Splunk's HTTP Event Collector (HEC)
- leveraging a highly distributed microservices based architecture on the backend
- chaining of reconnaissance tasks to enable full automation of testing all targets under a single root domain
- visualization of the data being sent into Splunk via the DarkTools Splunk App
The alpha release still has some drawbacks that I hope to address in the next version.
Please see our BlackHat presentation for more information behind this tool.
II. Usage
II. A. Quick Setup and Development
Example of Setting Up the Framework for Local or Development Usage
I tested this using a Kali Linux 2017.1 VM running within VMware Fusion
$ md5 Kali-Linux-2017.1-vm-amd64.7z
MD5 (Kali-Linux-2017.1-vm-amd64.7z) = 56b79dddb6f9c8ae71d2119c395ecd21
- Setup Splunk w/ HEC with an Index for "DarkTools"
- Install the Splunk "DarkTools" App
- Run the all in one script... ./run_all_with_console.sh
II. B. Production
Example of Building the Framework for Production Highly Distributed Usage
II. B. i. Supporting Systems
We need to get a few supporting systems in place for this to all work
II. B. i. a. Splunk
- You will need a Splunk Server with the HTTP Event Collector (HEC) Enabled.
- Create an index within Splunk called "darktools".
- Generate a HEC Key to be used with the DarkTools Framework.
- Ensure the microservices can send data to the HEC (e.g. AWS Security Groups ACL configuration)
II. B. i. a. Private Docker Registry
- Setup a Server
I deployed within AWS Ubuntu 16.04 x64 server onto a t2.small instance
AMI ID: ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170619.1 (ami-73f7da13)
- Configure ACLs
I reconfigured the security groups to all 22 from my IP with HTTP & HTTPS open to the servers with the microservices
- FQDNs
I pointed a subdomain to my EC2 instance (e.g. hub.darkgrifter.com)
- Install Docker w/ Let's Encrypt
chmod +x scripts/install_docker_ubuntu_16.sh
./scripts/install_docker_ubuntu_16.sh
docker pull registry:2
mkdir /opt/registryData
# Insecure # docker run -d -p 5000:5000 -v /opt/data:/var/lib/registry registry:2
cd /opt/
apt-get -y install git
git clone https://github.com/letsencrypt/letsencrypt.git
cd letsencrypt
./letsencrypt-auto certonly --keep-until-expiring --standalone -d hub.darkgrifter.com --email info@hub.darkgrifter.com
line="30 2 * * 1 /opt/letsencrypt/letsencrypt-auto renew >> /var/log/letsencrypt-renew.log"
(crontab -u root -l; echo "$line" ) | crontab -u root -
cd /etc/letsencrypt/live/hub.darkgrifter.com/
cp privkey.pem domain.key
cat cert.pem chain.pem > domain.crt
chmod 777 domain.crt
chmod 777 domain.key
docker run -d -p 443:5000 --restart=always --name registry \
-v /etc/letsencrypt/live/hub.darkgrifter.com:/certs \
-v /opt/registryData:/var/lib/registry \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
registry:2
See references section below for more information on how to set this up
II. B. ii. Build
- change into the Usage containing the code:
git clone ...
cd DarkToolsFramework
- edit configuration file:
vi shared/config.yaml
HEC_URL: 'https://ip.of.splunk.hec:8088/services/collector'
HEC_KEY: 'TOKEN-TOKEN-TOKEN-TOKEN-TOKEN'
AMQP_PROTO: 'amqp://'
AMQP_SERVER: 'ip.of.mq.microservice'
AMQP_PORT: '5672'
- optionally delete all docker containers on the build system:
./scripts/clean_docker_everything.sh
- build using the build script:
python pushDarkTools.py -e stage -r hub.darkgrifter.com
II. B. iii. Run the DarkTools Framework
Example of Running the Framework for Production Highly Distributed Usage
- I then create an SSH tunnel to my splunk server via a command like:
ssh -N -D 9050 -i ~/ssh_private_key.pem ubuntu@ip.of.splunk.server
- I then setup FoxyProxy within FireFox to use the SSH tunnel via an SOCKSv4 proxy on TCP port 9050
URL: https://127.0.0.1:8000/
Username: admin
Password: redacted_strong_pasword
III. References
- This blog inspired the overall design of the project...
http://brunorocha.org/python/microservices-with-python-rabbitmq-and-nameko.html
- Many StackOverflow articles helped out when I need a quick snippet of code, including but not limited to...
https://stackoverflow.com/questions/1885525/how-do-i-prompt-a-user-for-confirmation-in-bash-script
- Private Docker Registry
https://docs.docker.com/registry/deploying/
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-private-docker-registry-on-ubuntu-14-04
https://gist.github.com/PieterScheffers/63e4c2fd5553af8a35101b5e868a811e
https://www.digitalocean.com/community/tutorials/how-to-install-docker-compose-on-ubuntu-16-04
https://www.trainingdevops.com/insights-and-tutorials/deploying-docker-registry-with-let-s-encrypt-ssl-tls-certs