TestForge

An IntelliJ Platform plugin that generates unit tests for JVM (Kotlin/Java) and TypeScript/JavaScript classes — but instead of stopping at "a test exists," it resolves how each collaborator should be tested from your actual code, and refuses to hand you a test that doesn't prove anything.

Right-click a class → Generate Tests with TestForge... → review/adjust the proposed strategy → get a suite that compiled, ran, and passed against your current code.

Licensed under the MIT License.

Why

Right-click-to-generate is table stakes for AI test generators. TestForge earns its place on two things most of them skip:

1. Strategy resolution. TestForge inspects each collaborator your class depends on and proposes how to handle it in a test — Mock, H2 (real repository against an in-memory DB), Spy, or Real — with a reason ("@Repository (JPA) → real repo on H2"). You can override any decision before generation runs.
2. Proof over assumption. A generated test that merely compiles and passes isn't trusted. It's run through a static tautology detector (and, for Java SUTs, full PIT mutation testing) — a test that doesn't actually assert anything meaningful about your code gets flagged and held back from insertion rather than silently added to your suite.

Features

• Right-click generation for Kotlin, Java, TypeScript, and JavaScript classes, from the editor or the Project View.
• Per-collaborator strategy resolution (Mock / H2 / Spy / Real) with an editable, explained default for each dependency.
• Bounded, deterministic context slicing — the prompt is built from a token-budgeted slice of your code, not the whole file or project, and is reproducible for the same class + settings.
• Compile → run → repair loop: a generated test that doesn't compile or doesn't pass against your current code is repaired or dropped, never inserted.
• Quality gate: a static tautology detector runs on every generated suite; PIT mutation testing runs on Java SUTs when available. The UI only ever shows a mutation score that was actually measured.
• Additional instructions box in the Configure dialog, so you can steer a generation with free-text guidance before it runs.
• Retry with guidance in the results panel: if a test fails to compile or fails to pass, retry with added instructions without starting over.
• Two LLM backends: Anthropic direct, or any OpenAI-compatible endpoint (OpenAI itself, Azure OpenAI, or a self-hosted gateway like Ollama/vLLM) — configurable per-provider, complete with a Test Connection check.
• Keys never leave PasswordSafe. API keys are stored via the IntelliJ Platform's encrypted credential store, never written to settings XML, logs, or prompts.

Requirements

• IntelliJ IDEA 2024.2 (build 242) or newer — Community or Ultimate.
• JDK 21 to build the plugin itself (the IDE's own bundled JVM is fine at runtime).
• TypeScript/JavaScript generation requires the bundled JavaScript plugin, which ships with IntelliJ IDEA Ultimate. Under Community, TestForge still works for Kotlin/Java classes; TS/JS support is simply unavailable.
• An API key for Anthropic or an OpenAI-compatible provider.

Installation

TestForge isn't yet on the JetBrains Marketplace. To install it locally:

git clone https://github.com/vikaspad/TestForge.git
cd TestForge
./gradlew buildPlugin

This produces build/distributions/TestForge-<version>.zip. In your IDE, go to Settings/Preferences → Plugins → ⚙️ → Install Plugin from Disk... and select that zip.

Alternatively, run ./gradlew runIde to launch a sandboxed IDE instance with the plugin already installed, for trying it out without touching your main IDE.

Usage

1. Configure a provider once: Tools → TestForge: Configure LLM Provider... (or Settings → Tools → TestForge). Choose Anthropic or an OpenAI-compatible endpoint, enter a model/key, and hit Test Connection.
2. Right-click a Kotlin, Java, TypeScript, or JavaScript class in the editor or Project View and choose Generate Tests with TestForge....
3. Review the proposed strategy per collaborator (adjust any that don't fit), add any extra instructions you want the model to follow, and click Generate.
4. TestForge compiles and runs the generated suite against your current code, flags anything tautological or failing, and shows the rest in the results panel. If something didn't make it in, use Retry with guidance to steer a fresh attempt without leaving the panel.
5. Insert the suite into your test source root from the results panel.

Building from source

./gradlew runIde          # launch a sandboxed IDE with the plugin loaded
./gradlew buildPlugin      # produce build/distributions/TestForge-<version>.zip
./gradlew verifyPlugin     # IntelliJ Plugin Verifier against the configured target IDEs
./gradlew test             # unit + platform fixture tests

Architecture

Single Gradle module, organized by pipeline stage:

analysis/   SUT model, collaborator extraction, stereotype detection (JVM + TS/JS)
strategy/   Collaborator -> test strategy resolution (Mock / H2 / Spy / Real)
context/    Token-bounded dependency slice over the SUT's PSI graph
prompt/     Prompt assembly from the slice + resolved strategy plan
llm/        Provider-agnostic LLM client (Anthropic + OpenAI-compatible), key storage
generation/ Prompt -> generate -> parsed suite
verify/     Compile check, test run, bounded repair loop
quality/    Tautology detector + PIT mutation gate
ui/         Actions, Configure dialog, results tool window
insert/     PSI-safe insertion into the test source root
settings/   Persisted provider/model configuration

Every stage between "class you right-clicked" and "test in your suite" is designed around one rule: the LLM proposes, deterministic checks (compilation, test execution, mutation kill) decide.