Skip to content

CodeClone 1.2.1: security hardening, deterministic baseline checks, UX polish

Choose a tag to compare

@orenlab orenlab released this 03 Feb 11:25
· 55 commits to main since this release
3e47109

Overview

This release focuses on security hardening, robustness, and long-term maintainability.
No breaking API changes were introduced.

The goal of this release is to provide users with a safe, deterministic, and CI-friendly
tool suitable for security-sensitive and large-scale environments.

Security & Robustness

  • Path Traversal Protection
    Implemented strict path validation to prevent scanning outside the project root or
    accessing sensitive system directories, including macOS /private paths.

  • Cache Integrity Protection
    Added HMAC-SHA256 signing for cache files to prevent cache poisoning and detect tampering.

  • Parser Safety Limits
    Introduced AST parsing time limits to mitigate risks from pathological or adversarial inputs.

  • Resource Exhaustion Protection
    Enforced a maximum file size limit (10MB) and a maximum file count per scan to prevent
    excessive memory or CPU usage.

  • Structured Error Handling
    Introduced a dedicated exception hierarchy (ParseError, CacheError, etc.) and replaced
    broad exception handling with graceful, user-friendly failure reporting.

Performance Improvements

  • Optimized AST Normalization
    Replaced expensive deepcopy operations with in-place AST normalization, significantly
    reducing CPU and memory overhead.

  • Improved Memory Efficiency
    Added an LRU cache for file reading and optimized string concatenation during fingerprint
    generation.

  • HTML Report Memory Bounds
    HTML reports now read only the required line ranges instead of entire files, reducing peak
    memory usage on large codebases.

Architecture & Maintainability

  • Strict Type Safety
    Migrated all optional typing to Python 3.10+ | None syntax and achieved 100% mypy strict
    compliance.

  • Modular CFG Design
    Split CFG data structures and builder logic into separate modules (cfg_model.py and
    cfg.py) for improved clarity and extensibility.

  • Template Extraction
    Extracted HTML templates into a dedicated templates.py module.

  • Added a py.typed marker for downstream type checkers.

  • Added __slots__ to performance-critical classes to reduce per-object memory overhead.

CLI & User Experience

  • Added a sequential execution fallback when process pools are unavailable (for example, in
    restricted or sandboxed environments).
  • Emit clear, user-visible warnings when cache validation fails instead of silently ignoring
    corrupted state.
  • Hardened HTML report template to safely embed JavaScript template literals and aligned it
    with linting requirements.

Testing & Quality

  • Expanded unit and integration test coverage across the CLI, CFG construction, cache
    handling, scanner, and HTML reporting paths.
  • Added security regression tests for dot-dot traversal and symlinked sensitive directories.
  • Tightened cache mismatch assertions to verify full state reset.
  • Achieved and enforced 98%+ line coverage, with coverage configuration added to
    pyproject.toml.
  • Added GitHub Actions workflow with Python 3.10–3.14 test matrix, including ruff and
    mypy checks.
  • CI baseline enforcement now runs on a single pinned Python version to avoid AST dump
    differences across interpreter versions.

Python Version Consistency for Baseline Checks

Due to inherent differences in Python’s AST between interpreter versions, baseline
generation and verification must be performed using the same Python version.

The baseline file now stores the Python version (major.minor) used during generation.
When running with --fail-on-new, codeclone verifies that the current interpreter version
matches the baseline and exits with code 2 if they differ.

This design ensures deterministic and reproducible clone detection results while preserving
support for Python 3.10–3.14 across the test matrix.

Fixed

  • CFG Exception Handling
    Fixed incorrect control-flow linking for try/except blocks.

  • Pattern Matching Support
    Added missing structural handling for match/case statements in the CFG.

  • Block Detection Scaling
    Made MIN_LINE_DISTANCE dynamic based on block size to improve clone detection accuracy
    across differently sized functions.