Skip to content

feat: add benchmark suite for concurrent scans and report generation (#256)#296

Open
divyansha12 wants to merge 2 commits into
utksh1:mainfrom
divyansha12:feat/benchmark-suite-issue-256
Open

feat: add benchmark suite for concurrent scans and report generation (#256)#296
divyansha12 wants to merge 2 commits into
utksh1:mainfrom
divyansha12:feat/benchmark-suite-issue-256

Conversation

@divyansha12
Copy link
Copy Markdown

Description

Implements a production-grade, offline-first performance benchmark suite to measure, baseline, and prevent regressions in critical SecuScan performance areas. This PR covers:

  • Concurrent Task Creation: Throughput and P50/P95 latencies for task start operations.
  • Parser & normalizer throughput: Performance checks for report payload construction and finding normalization.
  • Database operations: Read latencies for fetching multiple rows and singular key lookups.
  • Export execution: CSV and HTML rendering latency performance.
  • SAT-3 slot saturation protection: Explicit resource safety test checking instantaneous reject times.
  • CLI benchmark runner: A standalone comparison script (scripts/run_benchmarks.py) that checks measurements against thresholds.json.

No third-party pip dependencies were added, relying solely on standard library utilities (time, statistics) and existing dev-requirements (pytest, pytest-asyncio).

Related Issues

Closes #256

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

How Has This Been Tested?

  • Benchmark Suite: Run locally with python3 scripts/run_benchmarks.py to confirm that all 11 performance metrics correctly query and pass against baseline thresholds.
  • Existing Test Verification: Run ./testing/test_python.sh to ensure all 349 existing test cases pass and benchmarks are correctly excluded from the main test suite.
  • Linting and Hygiene: Ran ruff check and git diff --check to ensure no trailing whitespace or lint warnings.

Checklist

  • My code follows the code style of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.

🟢 GSSoC note – I’m a GSSoC open source contributor. All checks are passing; please add the gssoc:approved label when merging.

…patibility

- Add new integration tests in testing/backend/integration/test_chaos_execution.py
- Fix Python version compatibility issues in testing/test_python.sh to use Python 3.11+
- Format codebase using black to comply with pre-commit static checks
- Recalculate and update integrity checksums for all 60 plugins
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PERF] Add benchmark suite for concurrent scans and report generation

1 participant