Skip to content

v0.5.6 — Large-Project Validation Suite & Rule Tuning Architecture

Choose a tag to compare

View all tags
@githubmofo githubmofo released this 27 Aug 06:30
· 38 commits to main since this release
v0.5.6
508b6ca

Release v0.5.6 — Large-Project Validation Suite & Rule Tuning Architecture

Date: 2026-08-27
Release Name: Large-Project Validation Suite & Rule Tuning Architecture
Tag: v0.5.6

🎯 Summary

TorusGuard v0.5.6 delivers a foundational leap in real-world validation, precision calibration, and transparent benchmarking.

While earlier versions established the 6-stage lifecycle, 60+ canonical security rules, and formal confidence scoring, v0.5.6 validates how the TorusGuard workflow performs across large, multi-module repositories with over 14,000+ files.

Under the core principle of Technical Honesty and Transparent Readiness, v0.5.6 introduces context-aware rule tuning guardrails to eliminate false positives without blind suppression, integrates seeded-case recall testing, establishes granular patch quality tracking, and formalizes honest readiness decisions that differentiate simulated dry-runs from real-world triage.

🚀 Key Highlights & Enhancements

1. 🏗️ 10 Large-Project Validation Architecture (harness/validate_large_projects.py)

  • Standardized multi-repository validation manifest (projects/manifest.yaml) targeting 10 production-scale Python repositories:
    • flask-cms (Flask)
    • django-core (Django)
    • fastapi-service (FastAPI)
    • sqlalchemy-orm (SQLAlchemy)
    • drf-api (Django & DRF)
    • sentry-multitenant (Django & React)
    • requests-client (Core Python)
    • pydantic-models (Pydantic)
    • celery-workers (Celery)
    • scrapy-spiders (Scrapy)
  • Totaling over 14,363 representative source files modeled and analyzed.

2. 🎯 Context-Aware Rule Tuning Guardrails

Tuned four foundational detection rules to properly account for framework-level abstractions:

  • TG-AUTH-008 (Untrusted Role/Tenant Headers): Distinguishes read-only metadata from authorization-influencing headers. Downgrades ambiguous external proxy/gateway contexts to Needs Review.
  • TG-INPUT-005 (Unsafe Template Rendering): Distinguishes framework-managed autoescaping (e.g. Jinja2/Django standard templates) from unescaped dynamic evaluation (render_template_string, mark_safe).
  • TG-INPUT-006 (Path Traversal): Avoids false alarms on benign os.path.join operations; escalates only when untrusted inputs reach storage without sanitization or allowlists.
  • TG-DB-004 (Missing Tenant Isolation): Recognizes tenant-scoped manager/repository layers, get_queryset() overrides, and dependency injection patterns.

3. 🧪 Seeded-Case Recall Measurement

  • Added known synthetic vulnerability seeds across target repositories in projects/manifest.yaml.
  • Establishes a formal recall measurement framework:
    $$\text{Recall} = \frac{\text{Detected Seeded Cases}}{\text{Detected Seeded Cases} + \text{Missed Seeded Cases}}$$
  • Prevents rule tuning from causing silent detection regressions.

4. 📊 Ponytail Patch Quality Evidence Ledger

  • Detailed tracking for every automated remediation patch:
    • File count and line churn ($+$lines / $-$lines).
    • Unrelated file modification checks (0 churn allowed).
    • Excess/unnecessary comment checks.
    • Automated project test and differential recheck assertion (Verified Fixed).

5. 🛡️ Transparent Pilot Readiness Decision

  • Strict policy against invented 100% metrics in simulated runs.
  • Clearly states the validation boundary:

    Final Readiness Decision: Not yet ready for controlled real-world use.
    The v0.5.6 harness successfully demonstrates the intended workflow design across 10 simulated large-project profiles. However, precision, seeded-case recall, false-positive counts, and patch safety remain unmeasured until actual repository scans and human triage are completed. Ready for a controlled pilot-validation phase only.

📚 New & Updated Artifacts

Assets 2
Loading