Skip to content

Releases

vibeaudit 0.11.1

Choose a tag to compare

@github-actions github-actions released this 18 Nov 04:13
e63f812

Bug fixes from an external review.

Fixed

  • vibeaudit test-exploit no longer gives an authentication verdict ("401 or 403", "2xx without
    authentication") for rules it has no automatic oracle for, such as BE006; it says to check the response by hand.
  • vibeaudit fix reads a Python file's indentation from its code, not from text inside strings, so a docstring or
    prompt indented by two spaces no longer makes it insert two-space blocks into a four-space file.
  • Python taint: mapping.get(key) takes its taint from the mapping and the default, not the key, so a user-chosen
    key into a server-side dict (FILES.get(request.args["name"])) is no longer a path traversal or injection source.
  • BE001 no longer reports f"... FROM {self.table}" when the class checked table with re.fullmatch against a
    pattern that only accepts identifiers before storing it.
  • The per-rule table in benchmark/RESULTS.md shows a stable rule's precision on the findings that gate, leaving out
    its findings in preview languages, as the headline numbers already did.
  • external in .vibeaudit.toml is read by the Python API's default options, the MCP server and vibeaudit watch.
    An inline comment had swallowed the key, so they always ran the external tools in auto mode; vibeaudit scan
    was not affected.

vibeaudit 0.11.0

Choose a tag to compare

@github-actions github-actions released this 17 Nov 12:24

Lexer-based scanners with labelled corpora for Rust, C# and PHP, which completes Phase 5 as preview; benchmark
numbers counted by each finding's own preview flag; and five rules back in preview after new held-out repositories.

Added

  • Rust (Actix, Axum): a lexer; attribute macros, .route builders, web::scope and Axum Router::route /
    .nest routes, with auth middleware (.wrap, .layer, .route_layer), guard attributes and auth extractors,
    including a project's own FromRequestParts extractor that rejects with 401/403; request data followed into SQL
    (sqlx, diesel, rusqlite), commands, outbound HTTP (reqwest), file paths and redirects; Cargo.toml dependencies
    locked from Cargo.lock and checked against OSV, offline bundle included.
  • C# (ASP.NET Core): a lexer (verbatim, interpolated and raw strings); controller and minimal-API routes with
    [Authorize] / [AllowAnonymous], RequireAuthorization(), route groups, fallback policies and a project's own
    auth filters; request data followed into SQL (EF Core raw SQL, ADO.NET, Dapper), commands, outbound HTTP, file
    paths and redirects; NuGet dependencies from .csproj and Directory.Packages.props checked against OSV. Three
    preview rules: CSHARP001 (no authorization anywhere in the project), CSHARP002 (deserialization that lets the
    payload pick the type: TypeNameHandling without a binder, BinaryFormatter), CSHARP003 (developer error page
    outside Development).
  • PHP (Laravel, Symfony): a lexer (inline HTML, heredoc and nowdoc); Laravel routes with groups, resources,
    middleware groups and controller middleware, and Symfony routes with #[IsGranted] and access_control; request
    data followed into SQL, commands, eval, unserialize, outbound HTTP, file paths, redirects and echoed HTML,
    including plain PHP pages; Blade {!! !!} output; Composer dependencies checked against OSV.
  • Benchmark: 11 Rust, 11 C# and 10 PHP repositories, every finding labelled, held-out repositories chosen and
    pinned before scanning (106 repositories in all).

Changed

  • Benchmark accounting: a finding counts as stable or preview by its own preview flag, not its rule's. Findings of
    a stable rule in Go, Java/Kotlin, C#, Rust or PHP code are preview and do not gate, so "stable" in the README and
    docs/BENCHMARK.md means exactly what --fail-on acts on. Stable precision is 94.4% tuned and 94.1% held-out.
  • Back to preview after new held-out repositories (STABILITY.md has the measurements): CFG002 (Unpinned container
    base image, 5 of 6), QA007 (Package imported but not declared as a dependency, 6 of 7), DATA012 (Foreign key
    without an index, 19 of 28), UI009 (Data loaded without loading or error state, 4 of 5) and QA001 (No automated
    tests, 6 of 8). None is a severe class. QA001's cause is fixed (tests in every supported language now count:
    _test.go, src/test, Rust test modules and files, .NET *.Tests projects, PHP); it returns to stable once
    measured on fresh held-out data.
  • AUTH002 and AUTH017 do not fire on an admin area of a project with no role concept anywhere, where every
    signed-in user is the admin.
  • Project-wide evidence (rate limiting, roles) reads Go and Rust code.
  • Fewer false positives, each from the tuned split: SEC003 on recorded API responses, translation folders, column
    names and validation strings; SEC011 on validation messages; FE002 and FE007 on placeholder URLs and server-render
    fallbacks; QA006 on export const enum; DATA001 on Rust impl ... for, #[cfg(test)] modules and PHP builder
    calls; CFG009 on env files that set no value.
  • C# HTTP clients are no longer reported for a missing timeout: .NET's HttpClient times out after 100 seconds.

vibeaudit 0.10.0

Choose a tag to compare

@github-actions github-actions released this 17 Nov 02:16

Five more languages in preview, with lexer-based scanners and labelled corpora for Go and Java/Kotlin, and the
first release published to PyPI from the release workflow.

Added

  • Go: a lexer; routes for net/http (including Go 1.22 "GET /path" patterns), Gin, Echo, Fiber and Chi, with
    groups, Route/Mount prefixes and auth middleware, feeding the shared route checks (AUTH001-AUTH003, AUTH010,
    LLM004); request data followed into SQL (database/sql, sqlx, GORM), commands, outbound HTTP, file paths and
    redirects; go.mod dependencies checked against OSV, offline bundle included. Four preview rules: GO001
    (goroutine started by a handler that never stops), GO002 (response body never closed), GO003 (map written
    concurrently without a lock), GO004 (SQL built with formatting).
  • Java and Kotlin (Spring Boot, Quarkus): a lexer; Spring @*Mapping and JAX-RS routes with the Spring Security
    chain (path and HTTP-method scoped matchers), Quarkus permissions and hand-written auth interceptors; request
    data followed into SQL, commands, deserialization, outbound HTTP, file paths and redirects; Maven and Gradle
    dependencies checked against OSV. Four preview rules: JAVA001 (controllers with no security configuration),
    JAVA002 (Actuator endpoints exposed), JAVA003 (security chain permits every request), JAVA004 (polymorphic
    deserialization without an allow-list).
  • C#, Rust and PHP: route and auth discovery, SQL and command injection, template XSS, N+1 queries in loops and
    HTTP clients without a timeout.
  • vibeaudit test-exploit: a proof-of-concept request for a finding, without destructive payloads. Dry run by default; --send
    goes to a loopback address only (checked after DNS resolution) and never follows redirects.
  • vibeaudit.runtime: FastAPI and Express middleware for staging, off unless VIBEAUDIT_RUNTIME=1 and refused in
    production.
  • openrouter as an LLM review provider. Keys are read from the environment only, never from a .env in the
    scanned project.
  • Benchmark: 10 Go and 10 Java/Kotlin repositories, every finding labelled, held-out repositories chosen and
    pinned before scanning.

Changed

  • Findings in Go, Java/Kotlin, C#, Rust and PHP are preview until measured on enough held-out findings
    (STABILITY.md); SQL injection, command injection and XSS still gate. Held-out so far: Java 10 of 12 true; Go
    GO004 2 of 2 and AUTH010 2 of 2.
  • QA006 falls back to packages when a tsconfig maps the catch-all "*" path (409 false positives on one Java
    project's front end).
  • AUTH010 reads rate limiting in Go code; BE018 recognises Go validation tags and libraries; _test.go files
    count as tests.
  • Installation: pip install vibeaudit from PyPI. Releases are published there by trusted publishing from the
    same attested wheel and sdist as the GitHub release.

vibeaudit 0.9.3

Choose a tag to compare

@github-actions github-actions released this 16 Nov 07:21

Full route paths behind router prefixes, UI checks for Vue and Svelte, and AUTH001 stable again.

Added

  • Router prefixes are resolved for FastAPI, Flask blueprints and Express routers, so vibeaudit routes, the route
    checks and the frontend-backend contract checks see full paths. The contract checks now also run in projects
    with prefixes, once every route's full path is known.
  • Vue and Svelte components get the same UI checks as React: keyboard access, labels, dead controls, index keys,
    unconfirmed destructive actions, cleanup of subscriptions and loading/error states.

Changed

  • AUTH001 (State-changing endpoint without authentication) is stable again: a route or server action that calls a
    dev-only guard that throws (assertDevelopment(), devOnly(), a NODE_ENV check that throws) is not reported.
    Held-out precision is 14 of 14 (it was 14 of 16 when it went back to preview in 0.9.2).
  • UI014 (Image without width and height) treats an <img> with a CSS-module class as sized by its stylesheet. It
    stays in preview: its held-out false positive has another cause.

vibeaudit 0.9.2

Choose a tag to compare

@github-actions github-actions released this 15 Nov 23:59

Cross-function taint for JavaScript and TypeScript, and two rules back in preview after a held-out measurement.

Added

  • JavaScript and TypeScript taint is followed into functions of the same file and named imports from project files
    (3 calls deep). A redirect to a URL an API client returned (such as a Stripe checkout session) is no longer
    reported as an open redirect.
  • Python taint is followed into the functions a route, agent tool or model call passes data to (same file, self
    methods and imported project modules, up to 3 calls deep), and findings say which call it came through. (This
    shipped in the 0.9.1 code; its note was missing from the 0.9.1 entry.)

Changed

  • Two rules go back to preview after a fresh held-out repository lowered their held-out precision below 90%
    (STABILITY.md):
    • UI014 (Image without width and height): 6 of 7 (85.7%); the false positive is an <img> sized by CSS classes.
    • AUTH001 (State-changing endpoint without authentication): 14 of 16 (87.5%); two server actions are guarded by
      assertDevelopment(). It keeps gating (gates_in_preview), like the other severe security rules: a missed
      unauthenticated write costs more than a false alarm.

vibeaudit 0.9.1

Choose a tag to compare

@github-actions github-actions released this 15 Nov 23:28

Credibility: measured recall and F1 next to precision, an honest list of misses, a comparison with Semgrep and
Bandit, and the means for others to check the labels.

Added

  • docs/BENCHMARK.md (python tools/benchmark.py report): precision, recall and F1 per category and per framework.
    Recall comes from 17 known vulnerable locations in the repositories pinned before a published advisory
    (benchmark/recall.json) and from the seeded cases; every advisory location vibeaudit misses is listed with the
    reason. Advisory recall is 35% (6 of 17).
  • A head-to-head with Semgrep (--config p/default, no login) and Bandit on the same pinned corpus
    (tools/compare.py, run by .github/workflows/compare.yml on Linux, manual and weekly): advisory recall,
    finding counts, overlap, and precision estimated from a labelled random sample of each tool's findings.
    First run: all three tools find the same 6 of the 17 advisory locations; precision is 88% for vibeaudit (every
    finding labelled), about 34% for Bandit and about 22% for Semgrep (50-finding samples, vibeaudit's rubric).
  • A sample report: .github/workflows/pages.yml publishes the HTML report and the six compliance reports for the
    deliberately vulnerable Next.js + Supabase fixture to GitHub Pages: https://ashin-06.github.io/vibeaudit/
  • A second-labeller kit (tools/labelling_kit.py, docs/LABELLING.md): a blind stratified sample of 150 labelled
    findings, labelling instructions, and Cohen's kappa against the maintainers' labels.
  • .github/workflows/publish.yml: trusted publishing of the attested wheel and sdist to PyPI, off until the
    one-time setup in docs/PUBLISHING.md is done.

Changed

  • The README's accuracy numbers are generated from the benchmark results by tools/benchmark.py baseline, and a
    test fails when they differ. They now say which numbers are stable rules only and which include preview rules
    (the held-out precision of stable rules is 95.7%; the README said 94%).

vibeaudit 0.9.0

Choose a tag to compare

@github-actions github-actions released this 13 Nov 01:41

Phases 0 to 4 of the roadmap are complete: a measured benchmark, the new checks, vibeaudit fix, the
integrations, and compliance mapping with the threat feed. The release passed the completion gate: a full audit
(docs/AUDIT-0.9.0.md) with every high and medium finding fixed, packaging checked in a
fresh environment on Python 3.9 to 3.13 on Linux, macOS and Windows, and a green benchmark gate.

Security

  • A project's llm setting is read only by vibeaudit scan: the LSP server, watch, studio, the MCP tools and the
    Python API never send code to a model provider because a repository's config asks for it.
  • vibeaudit studio binds its port exclusively, so on Windows no other local process can bind the same port and
    answer in its place.

Fixed

  • Files and messages nested too deeply to parse (package.json, tsconfig, pyproject, config, feedback and baseline
    files, LSP and MCP messages, bundles, the threat feed, cache files) are treated as malformed instead of crashing
    the scan or hanging the LSP server.
  • A damaged or forged bundle, a report path that cannot be written and an oversized studio query now fail with a
    clear error instead of a traceback.
  • The VS Code extension's README installs the language server from the repository at the matching version;
    vibeaudit is not on PyPI.

Changed

  • Five rules leave preview. Each has at least 5 held-out findings at 90% or more and has been released in
    preview at least once (STABILITY.md), and now counts toward --fail-on, the score and the
    grade:
    • CFG010 Third-party GitHub Action not pinned to a commit (held-out 24/24, tuned 65/65)
    • CFG016 Unencrypted database or disk storage (held-out 11/11, tuned 11/11)
    • DATA012 Foreign key without an index (held-out 9/9, tuned 70/70)
    • QA011 Same function copied into several files (held-out 8/8, tuned 14/14)
    • UI014 Image without width and height (held-out 6/6, tuned 16/17)
  • DATA013 (Redis key without an expiry) meets the held-out bar (6/6) but is 6 of 8 over the whole corpus
    (75%), so it stays in preview with that reason.

Added

  • Every rule is mapped to the OWASP Top 10 2021, the OWASP Top 10 for LLM Applications 2025, EU AI Act
    themes, SOC 2 and, where they apply, HIPAA and GDPR (data/compliance_map.json).
    --report compliance:FRAMEWORK writes a self-contained, printable HTML page with each control's status and
    links to the evidence. This is an aid for reviewers, not a certification or legal advice.
  • vibeaudit update downloads a signed threat feed: slopsquatted package names (DEP001, also offline), a model
    sunset calendar with retirement dates and replacements (LLM012), and prompt-injection patterns for agent
    instruction files (new preview rule LLM025). The same data ships with vibeaudit; a feed is used only when its
    signature verifies and it has not expired. See docs/THREAT_FEED.md.

vibeaudit 0.8.8

Choose a tag to compare

@github-actions github-actions released this 12 Nov 19:23

Phase 1 is complete: the last data, operations and CI checks.

Added

  • Four preview rules:
    • DATA014: personal data (email, phone, name, address, date of birth) stored in vector store metadata.
    • DATA015: a cache-aside read that recomputes on a miss with no lock or single-flight (cache stampede).
    • DATA016: a queue or scheduled job that charges, emails or inserts with nothing that makes a retry safe
      (no idempotency key, processed-job record, upsert or unique constraint).
    • CFG018: a pull_request_target workflow that checks out the pull request's code while secrets are
      available.

Fixed

  • Files with thousands of unclosed brackets no longer slow the JavaScript analysis down quadratically:
    bracket matching and statement ends now use one pairing pass per file (a hostile file took 30 to 90
    seconds, now a few). Results on well-formed code are unchanged, which tests check against the old scan.

vibeaudit 0.8.7

Choose a tag to compare

@github-actions github-actions released this 12 Nov 18:56

Phase 1 adds the remaining access-control checks.

Added

  • Four preview rules:
    • AUTH015: a password change or reset that keeps the other sessions and refresh tokens valid. Projects
      that revoke sessions or tokens somewhere, and frameworks that do it themselves (Django, Firebase,
      Clerk), are skipped.
    • AUTH016: a signed-in route that queries a table with a tenant column (org_id, team_id, workspace_id)
      without filtering by tenant. Supabase tables under row level security are skipped.
    • AUTH017: an admin, feature-flag or maintenance endpoint that any signed-in user can call (no role check).
      Endpoints with no authentication at all remain AUTH001.
    • FE008: a server action that returns a server secret, or an inline server action that closes over one.

vibeaudit 0.8.6

Choose a tag to compare

@github-actions github-actions released this 12 Nov 18:29

Phase 1 adds the remaining LLM checks; Phases 2 and 3 are complete with vibeaudit studio and the Redis TTL codemod.

Added

  • Six preview rules for LLM features:
    • LLM019: model output passed as an argument of a program run without a shell (git checkout <answer>),
      where a value such as --upload-pack=... becomes an option. Arguments after -- are not reported.
    • LLM020: an agent loop that calls the model for as long as it asks for tools, with no step limit.
    • LLM021: user text placed in a prompt without delimiters, so instructions inside it read like yours.
    • LLM022: web pages or files loaded into a vector store with no cleaning step (chunk poisoning).
    • LLM023: model calls for signed-in users with nothing tracking tokens or cost per user (apps where users
      bring their own API key are skipped).
    • LLM024: a project with tests, none of which touches its LLM code, and no eval setup.
  • Tests that pin down two Phase 1 items existing rules already cover: model output used as SQL, a shell
    command or a file path (LLM002), and "fetch this URL" features and redirects fed by request data (BE005,
    BE008).
  • vibeaudit studio: a local web UI (127.0.0.1 only, token-protected) with routes, data flows and a score
    drill-down.
  • vibeaudit fix adds an expiry to Redis keys (DATA013): ex=3600 / EX on set, or an
    expire(key, 3600) after hash, list, set and sorted-set writes.

Changed

  • The fuzz test's time budget uses the median of three calibration runs, so slow shared CI runners no longer
    fail it; it still catches super-linear paths.