-
Notifications
You must be signed in to change notification settings - Fork 597
Features
Below are a few helpful nuggets for getting started with the Recon-ng framework. While not all features are covered, the following notes will help make sense of a few of the frameworks more helpful and complex features.
- Interactive Help
- Command Completion
- Database Interaction
- Shell Commands
- Spooling Activity
- Recording Commands
- Configuration Persistence
- Global Options
- Workspaces
- Module Marketplace
- Module Searching
- Smart Loading
- Database Snapshots
- Restricted Context
- Module Details
- Data Sources
- Third Party Modules
- Methodology Driven
- Automation
- Analysis and Reporting
- Analytics
All commands and subcommands within the framework include interactive help documentation. For root commands, typing help <command> will provide this documentation, which includes documentation for invoking subcommands. For subcommands, providing the wrong input will prompt the framework to display specific subcommand level documentation. Use this interactive help documentation to learn the commands of the framework, what they do, and how to invoke them.
The entire framework is equipped with command completion. Whether exploring standard commands, or passing parameters to commands, tap the "tab" key several times to be presented with all of the available options for that command or parameter.
The db command provides an interface for the underlying database through several subcommands. The db query subcommand assists in managing and understanding the data stored in the database. Users are expected to know and understand Structured Query Language (SQL) in order to interact with the database via the db query command. The db schema command provides a graphical representation of the database schema to assist in building SQL queries. The db schema command creates the graphical representation dynamically, so as the schema of the database changes, so will the result of the command.
Every piece of information stored in the Recon-ng database is a potential input "seed" from which new information can be harvested. The db insert subcommand allows users to add initial records to the database which will become input for modules. Modules take the seed data, transform it into other data types, and store the data in the database as potential input for other modules. Database records can also be deleted using the db delete subcommand.
The shell command and ! alias give users the ability to run system commands on the local machine from within the framework. Using this method to run shell commands allows the framework to store the output if spooling is enabled.
A recorded session of all activity is essential for many penetration testers, but built-in OS tools like "tee" and "script" break needed functionality, like tab completion, and muck with output formatting. To solve this dilemma, the framework is equipped with the ability to spool all activity to a file for safe keeping. The spool command gives users the ability to start and stop spooling, or check the current spooling status. The destination file for the spooled data is set as a parameter of the spool start command, spool start <filename>. Use help spool for more information on the spool command.
Automating the framework using resource files is a powerful capability. To make it easy to create resource files, the framework is equipped with the ability to record commands. The script command gives users the ability to start and stop command recording, or check the current recording status. The destination file for the recorded commands is set as a parameter of the script record command, script record <filename>. Use help script for more information on the script command.
The modular nature of the framework requires frequently switching between modules and setting options unique to each one. It would be cumbersome to repeatedly set module options as information flows through the framework. Therefore, option values for all contexts within the framework are stored locally per workspace and loaded dynamically each time the context is loaded. This provides persistence to the configuration of the framework between sessions.
Pay attention to the global options, as they have changed over time and have a large impact on the performance of the framework. Many of the online tutorials regarding Recon-ng are outdated and misrepresent the purpose of global options as they stand now. Global options are the options that are available at the global context of the framework and have a global effect on how the framework operates. Global options such as "VERBOSITY" and "PROXY" drastically change how the modules present feedback and make web requests. Explore and understand the global options before diving into the modules.
Workspaces help users to conduct multiple simultaneous engagements without having to repeatedly configure global options or databases. All of the information for each workspace is stored in its own directory underneath the "~/.recon-ng/workspaces/" folder. Each workspace consists of its own instance of the Recon-ng database, a configuration file for the storage of configuration options, reports from reporting modules, and any loot that is gathered from other modules. To create a new workspace, use the workspaces command, workspaces create <name>. Loading an existing workspace is just as easy, workspaces load <name>. To view a list of available workspaces, see the workspaces list command. To remove a workspace, use the workspaces remove command, workspaces remove <name>. Workspaces can also be created or loaded at runtime by invoking the -w <workspace> argument when executing Recon-ng, ./recon-ng -w practisec.
The marketplace command provides the ability to search, analyze, install, and remove modules. When searching for modules using the marketplace search subcommand, it is important to take note of any modules that require dependencies or third party resource credentials as noted by asterisks in the D and K columns of the output. Use the marketplace info <module> subcommand to get a list of the required dependencies and/or credentials, as these will need to be installed and/or added manually prior to the module functioning properly. Installing a module before its dependencies have been met will result in error messages when loading, and the module being placed in "disabled" status. Loading a module before its credentials have been added will also result in a warning message about the key being missing, but the module will not be placed in "disabled" status and will likely fail at runtime due to the inability to authenticate properly to the resource.
The framework itself does not include any modules by default. Modules must be installed from the Module Marketplace. Use the marketplace install <module> subcommand to install a module, or the marketplace install all subcommand to install all modules. Caution, while installing all modules replicates the original state of older versions of Recon-ng, the framework will present the errors mentioned above for every module that is not fully configured, just like the older versions did. Picking and choosing which modules to install and configure is the beauty of the new Module Marketplace.
The marketplace search and modules search subcommands provide the capability to search the names of all available/installed modules and present the matches to the user. The search subcommand can be very helpful in determining what to do next with the information that has been harvested, or identifying what is required to get the desired information. The "recon" branch of the module tree follows the following path structure: recon/<input table>-<output table>/<module>. This provides simplicity in determining what module is available for the action the user wants to take next. To see all of the modules which accept a domain as input, provide the input table name "domains" followed by a dash: <marketplace|modules> search domains-. To see all of the modules which result in harvested hosts, provide the output table name "hosts" preceded by a dash: <marketplace|modules> search -hosts.
Even with command completion, module loading can be cumbersome because of the directory structure of the module tree. To make module loading easier, the framework is equipped with a smart loading feature. This feature allows modules to be loaded by referring to a keyword unique to the desired module's name. For instance, modules load namechk will load the "recon/contacts-contacts/namechk" module without requiring the full path since it is the only module containing the string "namechk". Attempting to smart load with a string that exists in more than one module name will result in a list of all possible modules for the given keyword. For example, there are many modules whose names contain the string "pwned". Therefore, the command modules load pwned would not load a module, but return a list of possible modules for the user to reference by full module name.
Backing up data at important points during the reconnaissance process helps to prevent the loss or corruption of data due to unexpected resource behavior. The snapshots command gives users the ability to backup and restore snapshots of the database. Use help snapshots for more information on the snapshots command.
After loading a module, the context of the framework changes, and a new set of commands and options are available. These commands and options are unique to the module. Use the help command to gain familiarity with the framework and available commands and options at the global and module contexts.
The info command is a helpful way to discover the capabilities of the currently loaded module. The info command returns detailed information about the loaded module including metadata, description, options, and any comments provided by the developer. Spend some time exploring modules with the info command to get a sense for what each module does.
Each module has a "SOURCE" option which determines the seed data (module input). The "SOURCE" option provides flexibility in what the user can provide to modules as input. The "SOURCE" option allows users to select "default" (seed data from the database as determined by the module developer), a single entry as a string, a path to a file, or a custom SQL query. The framework will detect the source and provide it as input to the module. The input command will list all of the expected input for the current configuration. Use the input command to ensure the "SOURCE" option configuration is providing the expected input. Changing the "SOURCE" option does not affect how the module handles the resulting information.
Users will likely create and share custom modules that are not merged into the module marketplace. In order to allow for the use of these modules without interfering with installed modules, create a uniquely named directory underneath the installed modules directory, i.e. "~/.recon-ng/modules/custom/". Modules added to this directory are loaded into the framework at runtime and listed under a category matching the folder's name, i.e. "Custom".
Modules are organized to facilitate the flow of a penetration test, and there are separate module branches within the module tree for each methodology step. Reconnaissance, Discovery, Exploitation and Reporting are steps 1, 3, 4 and 5 of the Web Application Penetration Testing Methodology. Therefore, each of these steps has their own branch in the module tree. It is important to understand the difference between Reconnaissance and Discovery. Reconnaissance is the use of open sources to gain information about a target, commonly referred to as "passive reconnaissance". Discovery, commonly referred to as "active reconnaissance", occurs when packets are explicitly sent to the target network in an attempt to "discover" vulnerabilities. While Recon-ng is a reconnaissance framework, elements from the other steps of the methodology will be included as a convenient place to leverage the power of Python.
The entire framework is scriptable through the use of a resource file. A resource file is a plain text file containing a list of commands for the framework. By referencing the resource file when executing Recon-ng, ./recon-ng -r <filename>, or issuing the script execute subcommand within the framework, script execute <filename>, the framework will read in the list of commands from the file and feed them to the command interpreter, in sequence. The resource file does not have to end by exiting the framework. The framework will automatically detect the end of the resource file and hand stdin back over to the terminal session for user input. The script is complete when the framework prompt looks like this: recon-ng > EOF.
If external shell scripting is preferred, the framework includes a tool called recon-cli which makes all of the functionality of the Recon-ng framework accessible from the command line. Use ./recon-cli -h for information on runtime options.
Probably the least known feature of Recon-ng is Recon-web. Recon-web is a web interface for analyzing and visualizing the data stored in the Recon-ng database, and exporting customizable data sets in various formats. Recon-web is invoked by running ./recon-web in the Recon-ng root directory. All of the functionality that exists in Recon-ng's reporting modules is available in Recon-web. Reporting modules should be considered deprecated and may eventually be completely phased out in favor of Recon-web.
When things break, as they often do when dealing with modules that harvest data from ever-changing web resources, someone is expected to fix it. The Recon-ng project has historically consisted of a one-man development team in terms of maintaining the framework, so an efficient means of determining how to prioritize maintenance tasks is paramount. Therefore, the Recon-ng framework contains an analytics engine that tracks the most commonly used modules. When a user reports a broken module, analytics data helps determine whether or not it is worth the effort to fix the module, find someone else to fix it, or remove it from the framework all together.
The first time Recon-ng runs, it creates a file in the user's home ~/.recon-ng directory called .cid containing a randomly generated UUID. This UUID is sent with each analytics request to differentiate users, allowing me to track how many different users are using the module. There is a big difference between one person using a module 3,000 times in a day, and 3,000 users using a module once a day. This information is critical to making good maintenance decisions. Analytics requests are sent each time a module is loaded using the modules load command. The analytics request includes the UUID, the module name, and the version of Recon-ng. For those that do not want to contribute, the entire system can be disabled by running Recon-ng with the --no-analytics flag. See the ./recon-ng -h help menu for more information.
Initially, a few users noticed this capability and complained, citing issues with custom modules being reported and not having the ability to disable the system completely. Both of these items have been addressed. Overall, it is a system that helps to more efficiently maintain the framework. The only thing attributable to the user is their IP address. To me, this is a non-issue. People use the Internet every day to visit web pages and logically attribute themselves to third parties. There is no targeting or harvested information included in the analytics. I encourage users to watch the traffic and validate for themselves.