Skip to content

AI in the Recon Pipeline

Jump to bottom
Samuele Giampieri edited this page Oct 7, 2026 · 9 revisions

📖 Canonical version: read this page on the official docs site — https://www.redamon.org/docs/ai-in-the-recon-pipeline. The GitHub wiki is a mirror.

AI in the Recon Pipeline

On this page

The idea: AI in small doses, not a black box

The loudest promise in AI offensive security is the fully autonomous agent: hand it a target, it reasons its way to a shell on its own. RedAmon ships that agent too, but it is not where most of the leverage lives. The reconnaissance pipeline is deterministic by design. The same target produces the same graph, the same scans fire in the same order, and every step is one you can read, test, and explain.

What this page documents is the other half of the strategy: injecting the LLM in small, well-scoped doses inside that deterministic pipeline rather than letting it drive the whole run. Each injection replaces one brittle heuristic (a static extension list, a hardcoded keyword match) with a narrow, tech-aware decision, then hands control straight back to the pipeline. The scan stays predictable and repeatable; only the point that needed judgement gets smarter.

The payoff is trust. An autonomous agent that guesses wrong leaves you without a thread to pull. A deterministic scanner with an AI-sharpened step gives you a result you can point at, reproduce, and improve one helper at a time.

How it differs from Adversarial AI Recon

Two things in RedAmon carry "AI" in their name and it is worth keeping them apart:

AI in the Recon Pipeline (this page) Adversarial AI Recon
Question it answers "How do I make my scanners smarter?" "Is the target running AI?"
What the LLM decides ffuf extensions, nuclei tags, WAF / takeover / false-positive calls no LLM decision at scan time; static signatures and active probes tag AI services on graph nodes
Where it lives recon/helpers/ai_planner/ decision helpers thin AI-aware hooks inside existing recon modules
Effect better scan inputs and fewer false positives ai_* / is_ai_* properties + Technology(category=ai-*) nodes

Same philosophy (AI in small doses inside deterministic modules), opposite direction: this page is the LLM improving the scan, Adversarial AI Recon is the scan discovering the target's AI attack surface.

The three contracts every helper obeys

Every helper under recon/helpers/ai_planner/ follows the same three rules. They are what make it safe to drop an LLM call into the middle of a live scan.

1. Never raise, always fall back

An AI helper is never allowed to break a scan. Every failure path (no API key, network timeout, malformed response, schema mismatch) returns the value the pipeline would have used without AI, never an empty result. Empty would silently turn a step off; falling back to the user's static configuration keeps the deterministic behaviour intact. Recon stdout also tails into the webapp's live recon drawer, so a raised exception would blank the stream as well as break the run.

2. Keys never enter the scan container

The recon image carries no LLM SDKs and no per-user API keys. Every helper POSTs its decision to an endpoint on the agent container (for example /llm/ffuf-extensions, /llm/nuclei-tags, /llm/waf-classify) which holds the credentials and talks to the provider. One credential boundary, one place to rotate keys, and a scan container that stays safe to spawn against a target. See AI Model Providers for where those keys are configured.

3. Cache by fingerprint, not by target

Where a decision depends on the tech stack, the helper caches on a fingerprint (the Server, X-Powered-By and similar headers) rather than the hostname. A hundred hosts behind one stack collapse to a single LLM call. A helper that only runs once per scan (the nuclei tag selector) needs no cache at all.

The five helpers

Helper Replaces Fires during Cache
FFuf extension planner static FFUF_EXTENSIONS list resource enumeration per-target (by header fingerprint)
Nuclei tag selector static NUCLEI_TAGS list vuln scanning setup once per scan
Nuclei false-positive response filter keyword is_false_positive() nuclei result triage per response
WAF classifier header-token WAF detection HTTP probing / security checks per response
Takeover false-positive classifier signature matching subdomain takeover triage per candidate

FFuf extension planner

The static extension list is a compromise: .php, .bak, .zip and a handful more, the same for every target. The planner reads the target's response headers (Server, X-Powered-By, X-AspNet-Version, ...) and asks the LLM for the extensions that actually match the stack, so an ASP.NET host gets .aspx / .asmx and a PHP host gets .php / .phtml instead of both getting the generic set. Falls back to a safe default list on any failure. Inspired by ffufai, reworked to keep keys out of the scan container.

Nuclei tag selector

Nuclei ships 9,000+ templates. Running all of them is slow; running a fixed tag subset misses stack-specific checks. The selector aggregates the http_probe fingerprint for the target (detected technologies plus Server headers) and asks the LLM for the tag set worth running, validated against the installed templates' TEMPLATES-STATS.json so it never selects a tag with no templates behind it. Runs once per scan, falls back to a broad candidate tag list.

Nuclei false-positive response filter

The static is_false_positive() check keys off a keyword list, which fails two ways: it fires on legitimate pages that merely contain the word "WAF" or "Access Denied" (admin panels, docs, API payloads), and it misses rebranded block pages that contain none of its keywords (custom Imperva themes, AWS WAF JSON, empty 406 bodies). When the keyword path is ambiguous, this helper hands the response body to the LLM to decide whether it is a genuine finding or a block page. Falls back to "not blocked" so a failed AI call never suppresses a real result.

WAF classifier

Modern WAFs strip or rebrand the headers the static _has_cdn_markers / check_waf_bypass logic looks for (cf-ray, Server: cloudflare, ...). The classifier scores WAF presence from the full response — status, headers, body sample, and response time — so it catches challenge-page body fingerprints, WAF cookie shapes (__cf_bm, incap_ses_), latency outliers, and status/body mismatches that no fixed token list can. Falls back to "no WAF detected".

Takeover false-positive classifier

A subdomain takeover candidate and a WAF block page can look identical: both can be a 404/403 whose body reads "resource not found", which also matches Heroku's "nothing here yet" and generic unclaimed-SaaS fingerprints. This classifier is invoked only in that ambiguous middle case — a candidate already flagged, with no clear vendor cookie or header — and asks the LLM whether the page is a real third-party unclaimed page or a WAF block, so a gated subdomain doesn't become a false takeover. Falls back to "not a WAF block" (keeps the candidate for review).

Turning it on

There is one master switch, AI in Pipeline (aiInPipeline), and one per-tool toggle for each helper (ffufAiExtensions, nucleiAiTags, nucleiAiResponseFilter, wafAiClassifier, takeoverAiClassifier). The master switch is a hard cascade, not just a convenience: turn it on and all five helpers are forced on, turn it off and all five are forced off. This is deliberate defence against drift between the master panel and the individual tool sections, which both bind to the same fields. Practically: flip AI in Pipeline in the project's Target section to control the whole set, and use each tool's own section (Nuclei, FFuf, and so on) to see the same state.

Every helper on the LLM engine shares one model, chosen by the AI Pipeline Model picker (aiPipelineModel); a hook you switch to TypeSafe Jev uses Jev instead. The recon container never holds the key: it delegates each call to the agent's /llm/* endpoints, so all five helpers use whichever provider that model id routes to. At least one AI Model Provider must be configured in Global Settings; with no provider (or the model left blank), the helpers fall back to their static behaviour and the scan runs exactly as it would without AI. See Project Settings Reference for every parameter.

These flags are also part of the AI-in-pipeline cascade, so an AI-generated Recon Preset can set them for you from a natural-language description of your goals.

Partial recon too. The helpers are wired into the shared entry functions, so they also fire when you re-run a single phase from the workflow graph rather than a full scan. The FFuf extension planner, for example, runs in both the full resource-enumeration phase and the standalone web-crawling partial-recon run.

Choosing the engine: LLM or TypeSafe Jev

Four of the five helpers can run on a second engine: TypeSafe AI Jev, a decision model that answers typed questions (yes/no, pick one, score) instead of writing text. It is fast and cheap, and its answers can only be one of the shapes your code asked for, which suits a decision buried inside a live scan.

The full guide, covering what Jev is asked for each hook, the thresholds, the failure table, cost and the MCP surface, is on its own page: TypeSafe Jev. This section is the summary.

Hook Engine field What Jev does
FFuf extension planner ffufAiUseJev Scores a fixed catalog of 40 extensions; never invents one. .bak and .old are always kept.
Nuclei tag selector nucleiTagsAiUseJev Scores each candidate tag. The universal tags (cve, exposure, misconfig, default-login, kev, oast, takeover) are always kept when present.
WAF classifier wafAiUseJev Answers "did a WAF or CDN edge produce this?" and picks one of 14 vendors.
Takeover classifier takeoverAiUseJev Answers "is this a WAF block page rather than the provider's unclaimed page?"

The Nuclei false-positive response filter has no Jev option, on purpose. It can delete a finding based on bytes the target controls, and Jev can be steered by crafted input, so that hook stays on the LLM.

Setting it up. Save one Jev API key on your account, either in Global Settings, LLM Providers, in the TypeSafe AI (Jev) section, or straight from the project form: the AI in Pipeline panel shows a Jev engine column next to the AI model, with the same key input when you have no token and a confirmation when you do. It is not a chat model and never appears in a model picker. Then, on each hook's card in that panel (or under the hook's toggle in its own tool section), pick LLM or Jev in the Engine row. The control is only enabled when the hook's AI toggle is on, and Jev can only be chosen when you have a token. Check API usage shows the key's status.

Three levels, not two. aiInPipeline is the master switch, each hook has its own AI flag, and the *UseJev field is the engine. The master switch forces the per-hook AI flags at scan start but never changes an engine choice, and applying a built-in preset keeps your engine choices too.

What contains a bad answer. Jev's answers are checked in code, not trusted: every answer set is closed (an extension or tag outside the catalog is impossible), the always-kept floors above hold whatever Jev says, and an answer can only tune, rank or annotate. A takeover finding is down-scored, never deleted. No finding or host is dropped because of a Jev answer.

When Jev is unavailable. A missing token, a rejected key, no credit or a TypeSafe outage makes the hook use its static fallback, the same coverage as a scan with no AI. It never silently re-routes to the LLM and never returns an empty result. A Jev failure is tracked separately from the LLM, so an empty Jev balance never switches off the LLM hooks for the rest of the run. Deleting your token mid-scan has the same effect on the next call.

Your data. Unlike the LLM path, which goes to whichever provider your model id routes to, the Jev path sends the response headers, body sample and tech fingerprint of the target to api.typesafe.ai. That makes TypeSafe a third party that receives target data when you turn the engine to Jev. The token itself stays in the agent container: recon only calls the agent and never sees it.

Jev-only hooks

Five more decisions have no LLM helper: they run only on Jev, and they were fixed steps before. Each asks Jev; a decision Jev cannot give falls back to the fixed behaviour. Four steps use the answer (page types, base paths, seed order, the serialized-object ranking); the tool-health check records its verdict next to the coverage gap.

Hook Setting What Jev is asked What the answer changes
Page-type labels httpxJevPageType Per probed page, one yes/no per class (login wall, parked, default install page, placeholder, error page); a deterministic pre-filter labels any page Jev gives no answer for A page_class label on each URL and on its Endpoint in the graph, app unless one class clears 0.70
FFuf base-path ranking ffufJevBasePaths Per discovered directory, "likely to hold sensitive, administrative or application content?" Which directories fill the smart-fuzz cap: FFuf fuzzes the ones Jev ranks highest (same number either way)
Hakrawler seed order hakrawlerJevSeedOrder Per probed host, "likely to have a rich web application surface?" The order Hakrawler crawls hosts in under its URL cap: the promising ones first (no seed dropped)
Tool health resourceEnumJevToolHealth Per empty collector result with unexplained error output, "a transient failure?" Jev's verdict is recorded next to the coverage gap; no tool is re-run
Serialized-object ranking serializedScanJevRank Per distinct blob the serialized-object scan flagged, "which format?" (one of 14) and "an attacker-reachable deserialization sink?" Jev's format and reachability land on each candidate (deser_jev_*), and the deserialization skill confirms the most reachable first; never drops a candidate. Needs serializedScanEnabled too

Each takes two settings: aiInPipeline and its own flag, which the master switch never sets or resets. Switching one on needs a Jev token on the project owner's account, like an engine switch. Each decision is also kept, next to the deterministic answer, in the recon output under jev_shadow.<hook>; the drawer gets at most 50 lines per hook and one summary. Under the tool-health hook there is a deterministic check that runs on every scan, Jev or not: an empty result from a crawler or collector that looks like a failure (a timeout, a crash, a non-zero exit, error output) is recorded as a coverage gap. See TypeSafe Jev for each hook in full.

Watching it work

Every helper logs to recon stdout with a bracketed tag so you can see the decisions in the live recon drawer:

  • [*][FFuf-AI] / [!][FFuf-AI]
  • [*][Nuclei-AI] / [!][Nuclei-AI]
  • [*][Nuclei-FP-AI] / [!][Nuclei-FP-AI]
  • [*][WAF-AI] / [!][WAF-AI]
  • [*][Takeover-AI] / [!][Takeover-AI]
  • [*][FFuf-Jev], [*][Nuclei-Jev], [*][WAF-Jev], [*][Takeover-Jev] (and their [!] forms) when a hook runs on Jev
  • [*][PageType-Jev], [*][FFuf-BasePaths-Jev], [*][CrawlOrder-Jev], [*][ToolHealth-Jev], [*][Serialized-Jev] and jev-shadow ... lines for the Jev-only hooks
  • [!][ToolHealth] when an empty collector result looks like a failure (always on)

[*] marks a decision the LLM (or Jev) made; [!] marks a fallback to the static value. Because every failure falls back, you can stop the agent container mid-scan and the recon run still completes on its deterministic path — the [!] lines simply tell you which steps ran without AI.

See also

RedAmon GitHub Repository | Report an Issue | Back to Home

Home

Getting Started

Core Workflow

Scanning & OSINT

AI & Automation

Plugins

HackLab

Analysis & Reporting

Contributing

Reference & Help

Clone this wiki locally