GitHub Actions: 
Project goals:
-
Distribution of network reconnaissance workload
- Scanning/reconnaissance is performed by set of agents allowing to perform pivoted scans and dynamic scheduling.
- Support for continuous scanning (periodic rescans)
-
Data analysis and management
- User-interface and API allows to analyze monitored infrastructure.
- ORM interface allows detailed automatic, semi-automatic or manual analysis.
-
reconnaissance
- agent -- modular wrapper for scanning tools
- scheduler -- job distribution
- planner -- management and scheduling for continuous scanning
-
data management
- parser -- agent outputs data parsing
- storage -- long term ip-centric storage
- visuals -- read-only analytics and visualization user-interface
- api -- REST-like api
+---+ (raw) files
agent +--+--+ server |
| |
+-------------+ | +--------------+ | +-----------------+
| | | | | | | | plugin1..N
| agent |<--------->| scheduler |---------->| parser |
| | | | | | | |
+-------------+ | +--------------+ | +-----------------+
| ^ queue1..N | |
plugin1..N | | + |
| | \|/
| +--------------+ +-----------------+
| | | | |
| | planner |---------->| db/storage |
| | | | |
| +--------------+ +-----------------+
| ^
| |
| +-----------------+
| | |
| | visuals |
| | |
| +-----------------+
+
- Flask-based web interface
- Authentication
- username/password with optional OTP
- FIDO2 / WebAuthn
- OIDC
- Per-route role-based authorization
- Server-side session storage (DEPRECATED)
- CLI interface and automation scripts
- Extendable plugin mechanims for extending agents and parsers
Agent provides communication and execution layer for plugins implementing various tools wrappers and supports fine-graned workload routing via capabilities metadata (DEPRECATED).
For list of currently available agent plugins see sner/plugin/*/agent.py
Scheduler provides workload configuration management and distribution with heatmap based rate-limiting scheduling.
-
Queue -- a list of targets and coresponding agent module and scheduling attributes container. Each module has a different config and target specification, see corresponding module implementation for details.
-
Job -- workload unit object, eg. assignment and output tuple. Job assignment features CIDR or regex targets exclusions based on configuration values.
-
Heatmap, Readynet -- internal structures for rate-limited target selection.
CLI helpers are available for IP ranges enumerations and queues/targets management.
Long-running daemon providing continuous orchestration of agents and output data processing in order to keep storage up-to-date with monitored networks reconnaissance data.
Storage is a main IP-centric database model and user interface heavily inspired by Metasploit framework PRO UI. Allows somewhat flexible data management including predefined aggregations and items tagging.
Tags are being evaluated in certain usecases:
-
i:anythingis ignored in vuln grouping (view and report generation). Tag is used to differentiate availability of the vuln/service/host/note from different scanning pivots (eg. i:via_externalnetwork, i:via_internalvpn), but visibility is ignored during report aggregations. -
report,report:data,infoare used to sort out issues already been processed by operator during engagement. Can be used to filter out and get "to be processed" vulns.
Parsers are used to parse and ingest agent output data or raw files to storage.
For list of currently available parser plugins see sner/plugin/*/parser.py
(EXPERIMENTAL) Pre-compiled view from various storage objects mapping product-version tuple to corresponding endpoints.
(EXPERIMENTAL) Pre-compiled view of CPE-CVE correlations.
Visualization modules for configuration and storage.
Basic access to managed data.
(EXPERIMENTAL) Set of external services for data analysis.
- Elastic storage (copy of storage data for analysis)
- CVE-Search (local instance used for CPE-CVE corelations)
# prepare environment
apt-get -y install git sudo make
git clone https://github.com/bodik/sner4 /opt/sner
cd /opt/sner
make install
# config and datastore
make install-db
editor /etc/sner.yaml
. venv/bin/activate && make db
# run prod server
apt-get -y install apache2 && a2enmod proxy proxy_http
cp extra/apache_proxy.conf /etc/apache2/conf-enabled/sner.conf
systemctl restart apache2
systemctl enable --now sner-server.service
# run agent
bin/server auth add-agent
editor /etc/sner.yaml
systemctl enable --now sner-agent@1.service
# run planner
systemctl enable --now sner-planner.service
# prepare environment
apt-get -y install git sudo make
git clone https://github.com/bodik/sner4 /opt/sner
cd /opt/sner
make githook
make install
make install-extra
make install-db
. venv/bin/activate
make db
# run tests
make lint
make coverage
make test-extra
# run dev server
bin/server run
- restart server with maintenance flag (
sner_maintenance: True) - wait for agents to finish
- stop agents, server and planner
- pull new version
- update dependencies
- perform db migrations
- start all components
- restart server without maintenance flag
systemctl stop sner-server
systemctl stop 'sner-agent@*'
cd /opt/sner
. venv/bin/activate
bin/server db stamp head
git fetch --all
git checkout origin/devel
pip install -r requirements.lock
bin/server db upgrade
systemctl start sner-server
- Generate target list
bin/server scheduler enumips 127.0.0.0/24 > targets1
bin/server scheduler rangetocidr 127.0.0.1 127.0.3.5 | bin/server scheduler enumips > targets2
- Enqueue targets in queue (web: scheduler > queue > enqueue)
bin/server scheduler queue-enqueue <queue.name> --file=targets
- Run the agent
- Monitor the queue until all jobs has been finished
- Stop the agent
bin/agent --shutdown [PID] - Gather recon data from queue directories (
<SNER_VAR>/scheduler/queue-<queue.id>)
- Import existing data with suitable parser
bin/server storage import <parser name> <filename>
- Use web interface, flask shell or raw database to consult or manage gathered data
- Generate preliminary vulnerability report (web: storage > vulns > Generate report)
nmap -sL 192.168.2.0/24 -oA scan-dnsenum
bin/server storage import nmap scan-dnsenum.xml
bin/server scheduler enumips 192.0.2.0/24 | bin/server scheduler queue-enqueue 'sner servicedisco nmap'
bin/agent --debug
bin/server storage import nmap /var/lib/sner/scheduler/queue-<queue.id>/*
# template
nmap SCANTYPE TIMING OUTPUT TARGETS
# example
bin/server storage service-list --filter 'Service.port == 22' --short > targets
nmap \
-sV --version-intensity 4 -Pn \
--max-retries 3 --script-timeout 30m --max-hostgroup 1 --max-rate 1 --scan-delay 10 \
-oA output --reason \
-p T:22 -iL targets
# import data
bin/server storage import nmap output.xml
See scripts/.
- Swagger UI does not work well for session authenticated users. In order to prevent CSRF for API endpoints only apikey must be used in the request. Use private-browser window.