feat(ENG-12397): remove legacy MLP inference mode by hiskudin · Pull Request #23 · StackOneHQ/defender

hiskudin · 2026-03-17T16:15:00Z

Summary

Removes the legacy MLP inference mode (frozen all-MiniLM-L6-v2 embeddings + JS MLP head) that predates the fine-tuned ONNX model
ONNX is now the only Tier 2 inference path — simpler, more accurate
Drops 3.4MB of bundled mlp_weights.json from the package

Files removed

src/classifiers/mlp.ts — MLP forward pass (pure JS matrix ops)
src/classifiers/weights.ts — MLP_WEIGHTS export
src/classifiers/embedder.ts — generic MiniLM embedder (Transformers.js)
src/classifiers/mlp_weights.json — 3.4MB pre-trained weights

Breaking changes

MLP_WEIGHTS no longer exported from the package
PromptDefenseOptions.tier2Weights removed
PromptDefense.loadTier2Weights() removed
Tier2ClassifierConfig.mode removed (ONNX is now the only mode)

Test plan

npm run test:typecheck passes
npm test — all 169 tests pass

🤖 Generated with Claude Code

Summary by cubic

Removed the legacy MLP inference mode and the per-field async Tier 2 path; ONNX is now the only Tier 2 classifier (ENG-12397). This trims ~3.4MB and surfaces ONNX errors in both single- and sentence-level classification.

Refactors
- Deleted src/classifiers/mlp.ts, embedder.ts, weights.ts, and mlp_weights.json.
- Simplified Tier2Classifier to ONNX-only; removed embedder/weights APIs and mode checks; updated tests and README.
- Removed dead Tier 2 async path from ToolResultSanitizer; sanitizer is now synchronous Tier 1-only.
- Removed ?? 0 fallbacks and use an instanceof Error check so inference errors propagate; skipReason now reports real errors in both classify() and sentence-level analysis.
Migration
- Remove mode from createTier2Classifier and tier2Config (ONNX is implicit).
- Remove MLP_WEIGHTS exports and any loadTier2Weights() calls; drop PromptDefenseOptions.tier2Weights.
- In ToolResultSanitizer usage, drop useTier2Classification, tier2Config, tier2Weights, warmupTier2(), isTier2Ready(), and any sanitizeAsync calls.

^{Written for commit 75cae15. Summary will update on new commits.}

The MLP mode (frozen embeddings + JS MLP head) was added before the fine-tuned ONNX model existed. Now that ONNX is the only mode, remove all MLP-specific code and the 3.4MB bundled weights file. Removes: mlp.ts, weights.ts, embedder.ts, mlp_weights.json Updates: tier2-classifier.ts, tool-result-sanitizer.ts, prompt-defense.ts, types.ts, config.ts, index.ts, specs/tier2-classifier.spec.ts, specs/onnx-classifier.spec.ts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR removes the legacy Tier 2 “MLP inference mode” (frozen MiniLM embeddings + JS MLP head + bundled weights) and makes the fine-tuned ONNX MiniLM pipeline the only Tier 2 inference path, reducing package size and simplifying the Tier 2 API.

Changes:

Remove Tier 2 “mode” switching and all MLP-related public APIs/options (weights export, weight-loading helpers, config fields).
Simplify Tier2Classifier to be ONNX-only and delete the legacy embedder/MLP implementation files.
Update tests to cover the ONNX-only Tier2Classifier behavior and remove MLP-specific test coverage.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/types.ts	Removes `tier2.mode` from public configuration types.
src/index.ts	Stops exporting `MLP_WEIGHTS` from the package entrypoint.
src/core/tool-result-sanitizer.ts	Removes Tier 2 weight plumbing and `loadTier2Weights()` in favor of ONNX-only Tier 2 initialization.
src/core/prompt-defense.ts	Removes `tier2Weights` option and `loadTier2Weights()` method; Tier 2 is now ONNX-only.
src/config.ts	Removes default Tier 2 `mode` setting (no longer applicable).
src/classifiers/tier2-classifier.ts	Deletes MLP/embedder branches; Tier 2 classifier now always uses ONNX path.
src/classifiers/index.ts	Removes exports for embedder/MLP/weights; keeps ONNX-based exports.
src/classifiers/weights.ts	Removed legacy bundled weight export module.
src/classifiers/mlp.ts	Removed legacy JS MLP implementation.
src/classifiers/embedder.ts	Removed legacy Transformers.js embedder used by MLP mode.
specs/tier2-classifier.spec.ts	Removes MLP tests; adds ONNX-only Tier2Classifier tests and sanitizer integration expectations.
specs/onnx-classifier.spec.ts	Updates Tier2Classifier tests to remove `mode: 'onnx'` usage and deletes mode-specific assertions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

src/classifiers/tier2-classifier.ts

cubic-dev-ai

2 issues found across 13 files

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="specs/tier2-classifier.spec.ts">

<violation number="1" location="specs/tier2-classifier.spec.ts:28">
P2: Skipping this test on CI removes the last CI coverage of the only Tier 2 inference path.</violation>

<violation number="2" location="specs/tier2-classifier.spec.ts:58">
P2: This assertion doesn't verify that Tier 2 was actually initialized.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

specs/tier2-classifier.spec.ts

The sanitizeAsync() path and helpers (sanitizeObjectAsync, sanitizeArrayAsync, sanitizeStringFieldAsync, etc.) implemented per-field Tier 2 classification during traversal. This was never reachable: PromptDefense always calls sanitize() (sync) with useTier2Classification hardcoded to false. Tier 2 runs at the PromptDefense level on combined text via classifyBySentence(), which also detects cross-field fragmented injections. ToolResultSanitizer is now a clean, synchronous Tier 1-only unit. Also removes MLP references from README. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Bare catch {} silently swallowed ONNX inference errors, causing all-sentence failures to return skipReason: "No classifiable sentences" — misleading when the real cause was a runtime error. Now captures the last error and surfaces it in skipReason when no sentences were successfully scored. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Follows repo convention of fully isolated, explicit tests without shared lifecycle hooks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Removes the legacy Tier 2 MLP inference path (frozen embeddings + JS MLP head) so Tier 2 inference is ONNX-only, shrinking the package and simplifying the API surface.

Changes:

Deleted legacy MLP implementation files (MLP forward pass, embedder, bundled weights JSON) and removed related exports/APIs.
Refactored Tier 2 classifier/config to be ONNX-only and updated PromptDefense + sanitizer integration accordingly.
Updated docs and tests to reflect the ONNX-only Tier 2 path and synchronous Tier 1-only tool-result sanitization.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/types.ts	Removes Tier 2 `mode` from config types to match ONNX-only behavior.
src/index.ts	Drops `MLP_WEIGHTS` export from the public API.
src/core/tool-result-sanitizer.ts	Removes Tier 2/async sanitizer path; sanitizer is now synchronous Tier 1-only.
src/core/prompt-defense.ts	Removes Tier 2 weights-loading option/API; Tier 2 classifier initialization is ONNX-only.
src/config.ts	Removes default Tier 2 `mode` setting.
src/classifiers/weights.ts	Removes bundled MLP weights export (file deleted).
src/classifiers/tier2-classifier.ts	Simplifies Tier 2 classifier to ONNX-only; improves sentence-level error surfacing.
src/classifiers/mlp.ts	Removes JS MLP implementation (file deleted).
src/classifiers/index.ts	Stops exporting legacy embedder/MLP/weights modules.
src/classifiers/embedder.ts	Removes Transformers.js embedder used by legacy MLP mode (file deleted).
specs/tier2-classifier.spec.ts	Removes MLP-focused tests; updates Tier 2 tests to ONNX-only + sanitizer sync integration.
specs/onnx-classifier.spec.ts	Updates Tier2Classifier ONNX tests for ONNX-only config surface.
README.md	Removes MLP mode documentation; clarifies ONNX-only Tier 2 setup.

Comments suppressed due to low confidence (1)

src/core/prompt-defense.ts:193

The docs around warmupTier2() / isTier2Ready() still refer to an "embedding model download (~30MB)" and "weights loaded". With the legacy MLP/weights path removed and Tier 2 now ONNX-only + bundled, these comments are misleading. Please update the wording to describe warming up/loading the bundled ONNX model + tokenizer (and remove references to weights/downloads).

	/**
	 * Pre-load the Tier 2 embedding model
	 *
	 * Call this at startup to avoid latency on first classification.
	 * The embedding model download (~30MB) is cached locally.
	 */
	async warmupTier2(): Promise<void> {
		if (this.tier2Classifier) {
			await this.tier2Classifier.warmup();
		}
	}

	/**
	 * Check if Tier 2 is ready (weights loaded and classifier available)
	 */
	isTier2Ready(): boolean {
		return this.tier2Classifier?.isReady() ?? false;
	}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

src/classifiers/tier2-classifier.ts

src/core/tool-result-sanitizer.ts

src/classifiers/tier2-classifier.ts

OnnxClassifier.classify() is typed to always return a number and throws on failure, so ?? 0 silently converts errors into safe scores. Removed both occurrences to let errors propagate to the catch/skipReason path. Also removed stale "Tier 2 MLP classifier" step from ToolResultSanitizer doc comment — it is now Tier 1 only. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Removes the legacy Tier 2 “MLP + frozen embeddings” inference path and related public APIs, making the fine-tuned ONNX model the only Tier 2 classifier implementation and reducing package size.

Changes:

Deleted legacy MLP/embedder codepaths and removed bundled mlp_weights.json + MLP_WEIGHTS export.
Simplified Tier2Classifier config/API to be ONNX-only (removed mode, weights loading, and embedder accessors).
Removed Tier 2 support from ToolResultSanitizer (now synchronous Tier 1-only) and updated tests/docs accordingly.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/types.ts	Removes `tier2.mode` from public config types.
src/index.ts	Drops the `MLP_WEIGHTS` export from the package entrypoint.
src/core/tool-result-sanitizer.ts	Removes async Tier 2 sanitization path and Tier 2 config/weights plumbing.
src/core/prompt-defense.ts	Removes Tier 2 weight-loading option/method and keeps ONNX Tier 2 initialization only.
src/config.ts	Removes default Tier 2 `mode` field to match ONNX-only behavior.
src/classifiers/weights.ts	Deleted: pre-bundled MLP weights export is removed.
src/classifiers/tier2-classifier.ts	Refactors Tier 2 classifier to ONNX-only; improves sentence-level error reporting.
src/classifiers/mlp.ts	Deleted: legacy JS MLP forward pass and weight loader removed.
src/classifiers/index.ts	Stops exporting embedder/MLP/weights; updates module docs to ONNX-only.
src/classifiers/embedder.ts	Deleted: Transformers.js embedder removed.
specs/tier2-classifier.spec.ts	Replaces MLP-focused tests with ONNX-only classifier + sanitizer integration tests.
specs/onnx-classifier.spec.ts	Updates ONNX test setup to remove `mode: 'onnx'` and removes deleted API assertions.
README.md	Updates documentation to remove MLP mode references and describe ONNX-only Tier 2.

Comments suppressed due to low confidence (1)

src/core/prompt-defense.ts:181

The warmupTier2() JSDoc still refers to pre-loading an "embedding model" and a ~30MB download. Tier 2 is now ONNX-only with a bundled model/tokenizer, so this description is misleading; update the comment to reflect ONNX warmup (and remove the download reference).

	/**
	 * Pre-load the Tier 2 embedding model
	 *
	 * Call this at startup to avoid latency on first classification.
	 * The embedding model download (~30MB) is cached locally.
	 */

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

src/classifiers/tier2-classifier.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Removes the legacy “frozen embeddings + JS MLP head” Tier 2 inference path and simplifies the library so Tier 2 is ONNX-only, reducing package size and API surface.

Changes:

Deleted the legacy MLP/embedder implementation and pre-bundled weights export (MLP_WEIGHTS).
Simplified Tier 2 configuration/API to ONNX-only (removed mode, weights-loading APIs, and Tier 2 async sanitizer path).
Updated tests and README to reflect the new ONNX-only Tier 2 behavior.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/types.ts	Removes Tier 2 `mode` from public config types.
src/index.ts	Removes `MLP_WEIGHTS` from top-level exports.
src/core/tool-result-sanitizer.ts	Removes Tier 2 async path and Tier 2-related configuration; sanitizer is now sync Tier 1-only.
src/core/prompt-defense.ts	Removes Tier 2 weights option / loader; updates Tier 2 warmup docs for ONNX-only flow.
src/config.ts	Removes default Tier 2 `mode` from config defaults.
src/classifiers/weights.ts	Deleted legacy `MLP_WEIGHTS` bundling module.
src/classifiers/tier2-classifier.ts	Refactors Tier 2 classifier to ONNX-only; improves error reporting in sentence classification.
src/classifiers/mlp.ts	Deleted legacy JS MLP inference implementation.
src/classifiers/index.ts	Stops exporting embedder/MLP/weights; updates Tier 2 description to ONNX.
src/classifiers/embedder.ts	Deleted legacy Transformers.js embedder used by MLP mode.
specs/tier2-classifier.spec.ts	Replaces MLP-focused tests with ONNX-only Tier2Classifier tests + sanitizer integration.
specs/onnx-classifier.spec.ts	Removes `mode: 'onnx'` and legacy ONNX-mode-only assertions (no-op weights/getEmbedder).
README.md	Removes MLP mode documentation; updates Tier 2 setup docs to ONNX-only.

Comments suppressed due to low confidence (1)

src/config.ts:175

skipBelowSize remains in the default Tier 2 config, but it is not referenced anywhere in the codebase (Tier 2 execution in PromptDefense only checks combinedText.length > 0). This makes the option misleading for users and dead config to maintain. Either enforce skipBelowSize when deciding whether to run Tier 2 (and set an appropriate tier2SkipReason), or remove it from PromptDefenseConfig/defaults if it’s no longer supported.

export const DEFAULT_TIER2_CONFIG = {
	highRiskThreshold: 0.8,
	mediumRiskThreshold: 0.5,
	skipBelowSize: 50, // Skip Tier 2 for very short strings
};

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

src/classifiers/tier2-classifier.ts

 			try {
-				let score: number;
-
-				if (this.config.mode === "onnx") {
-					score = (await this.onnxClassifier?.classify(sentence)) ?? 0;
-				} else {
-					const embedding = await this.embedder?.embedOne(sentence);
-					if (!this.model || !embedding) {
-						throw new Error("MLP model or embedder not available");
-					}
-					score = mlpForward(this.model, embedding);
-				}
+				const score = await this.onnxClassifier.classify(sentence);



Copilot AI review requested due to automatic review settings March 17, 2026 16:15

Copilot started reviewing on behalf of hiskudin March 17, 2026 16:15 View session

hiskudin changed the title ~~feat: remove legacy MLP inference mode~~ feat(ENG-12397): remove legacy MLP inference mode Mar 17, 2026

Copilot AI reviewed Mar 17, 2026

View reviewed changes

src/classifiers/tier2-classifier.ts Show resolved Hide resolved

cubic-dev-ai bot reviewed Mar 17, 2026

View reviewed changes

specs/tier2-classifier.spec.ts Outdated Show resolved Hide resolved

specs/tier2-classifier.spec.ts Outdated Show resolved Hide resolved

hiskudin and others added 3 commits March 17, 2026 16:47

test: remove beforeEach, instantiate classifier per test

bfd1ac8

Follows repo convention of fully isolated, explicit tests without shared lifecycle hooks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

hiskudin requested a review from Copilot March 17, 2026 16:55

Copilot started reviewing on behalf of hiskudin March 17, 2026 16:56 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

src/classifiers/tier2-classifier.ts Outdated Show resolved Hide resolved

src/core/tool-result-sanitizer.ts Show resolved Hide resolved

src/classifiers/tier2-classifier.ts Outdated Show resolved Hide resolved

hiskudin requested a review from Copilot March 17, 2026 17:09

Copilot started reviewing on behalf of hiskudin March 17, 2026 17:10 View session

test(defender): split multi-assertion tests into individual cases

7e77542

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI reviewed Mar 17, 2026

View reviewed changes

src/classifiers/tier2-classifier.ts Show resolved Hide resolved

src/classifiers/tier2-classifier.ts Show resolved Hide resolved

hiskudin and others added 2 commits March 17, 2026 17:22

fix(defender): use instanceof check in classify() skipReason

0ef9c98

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test(defender): split assertions and add arrange/act/assert comments

75cae15

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

hiskudin requested a review from Copilot March 17, 2026 17:34

Copilot started reviewing on behalf of hiskudin March 17, 2026 17:34 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ENG-12397): remove legacy MLP inference mode#23

feat(ENG-12397): remove legacy MLP inference mode#23
hiskudin wants to merge 8 commits intomainfrom
feat/remove-mlp-legacy

hiskudin commented Mar 17, 2026 •

edited by cubic-dev-ai bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

cubic-dev-ai bot left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

hiskudin commented Mar 17, 2026 • edited by cubic-dev-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Files removed

Breaking changes

Test plan

Summary by cubic

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hiskudin commented Mar 17, 2026 •

edited by cubic-dev-ai bot

Loading