Skip to content

v0.7.0: Offline Dataset Architecture, Modular Tool Engine & CI Security Hardening

Choose a tag to compare

@rehan1020 rehan1020 released this 30 Oct 13:54
· 34 commits to main since this release
d9d6037

v0.7.0: Offline Dataset Architecture, Modular Tool Engine, CI Security Hardening & Cloud Deploy Safeguards

v0.7.0 delivers a comprehensive architectural upgrade to mcp-india-stack:

  1. Offline-First Dataset Architecture (Update 6.7): Eliminates runtime network downloads, caching, and background daemon threads in favor of immutable offline bundled datasets.
  2. Automated CI Security Pipeline & Regression Guards (Update 7): Introduces Bandit SAST, pip-audit vulnerability scanning, lockfile drift detection, distribution wheel smoke testing, immutable GitHub Action SHA pinning, and 20 architectural regression guards (G01–G20).
  3. Modular Tool Engine & 100% Core Parity (Update 5b): Extracts all remaining inline registration tools into dedicated modules under tools/, converts registration wrappers to thin exception delegates, and achieves 100% core function parity verified by golden output testing.
  4. Cloud & Web Transport Resiliency: Adds public unauthenticated /health endpoints and automatic 0.0.0.0 interface binding on cloud PaaS environments (e.g. Render) to ensure zero-downtime health checking and prevent port scan timeouts.

📦 What's New in v0.7.0

1. Offline Bundled Datasets (Update 6.7)

  • Zero Runtime Dataset Downloads: Removed dynamic HTTP/CDN dataset downloads, background updater threads, locking mechanisms, and local cache-directory resolution (MCP_INDIA_STACK_CACHE_DIR).
  • Immutable Offline Bundle: All lookup datasets (IFSC index, PIN codes, HSN codes, court establishment codes) are now bundled directly within the package wheel with cryptographic SHA-256 integrity verification.
  • CLI --refresh-all Deprecation: Deprecated and hidden (help=argparse.SUPPRESS). Invoking --refresh-all prints a clear stderr notice advising users to update the package and exits with code 2 without performing socket or network I/O.
  • Maintainer Verification Tooling: scripts/update_datasets.py is now a maintainer-only offline utility supporting --write-checksums to generate dataset_checksums.json against bundled files.
  • Git Binary Protection: Added .gitattributes rule enforcing binary treatment on CSV and compressed data files to prevent CRLF line-ending mutations across operating systems.

2. CI Security Gates & Invariant Regression Guards (Update 7)

  • Bandit SAST Scanning: Added automated AST security static analysis in CI (bandit -r src/ -ll -ii) catching potential vulnerabilities prior to merge.
  • Automated Dependency Auditing (pip-audit): CI enforces automated vulnerability scanning against OSV databases with strict expiry date tracking via .pip-audit-ignore.toml.
  • Lockfile Drift Checks: Added scripts/check_lock.py to prevent silent drift between pyproject.toml and compiled requirements.txt.
  • Wheel Smoke Testing: Added scripts/smoke_wheel.py to build the distribution wheel, install it in a clean isolated virtual environment, and verify CLI entrypoints, tool counts, and transport security.
  • Immutable GitHub Actions Pinning: Pinned all GitHub Actions across CI, Publish, and Security Audit workflows to immutable 40-character commit SHAs.
  • 20 Architectural Regression Guards (G01–G20): Added tests/test_regression_guards.py verifying critical invariants on every pull request:
    • G01: CORS allowlist and credential protection
    • G02: Constant-time token comparisons (hmac.compare_digest)
    • G03: Worker clamping bounds [1, 20]
    • G04: Dataset SHA-256 integrity
    • G05: Zero network client imports in utils/
    • G06: Dead database code elimination
    • G07: Telemetry HMAC pseudonymization
    • G08: Elevated tool gating (MCP_INDIA_STACK_ENABLE_ELEVATED_TOOLS=1)
    • G09: Normalization AST imports
    • G10: server.py line count bounds ($\le 705$ lines)
    • G11: Strict AST delegation across all tool wrappers
    • G12: Single isolated yfinance import
    • G13: Loopback host defaults (127.0.0.1)
    • G14: Starlette middleware execution order
    • G15: Startup refusal matrix for weak keys and exposed interfaces
    • G16: Host and origin allowlist enforcement
    • G17: Failed auth rate limit tracking
    • G18: Bounded memory stores
    • G19: Version tagging verification
    • G20: Zero skipped or xfailed tests

3. Modular Tool Engine & Core Parity (Update 5b)

  • 9 Extracted Core Tools: Inline tools formerly embedded directly in registration files were refactored into dedicated modules under src/mcp_india_stack/tools/:
    • bulk_validate_gstin → tools/bulk_gstin.py
    • bulk_validate_pan → tools/bulk_pan.py
    • bulk_validate_ifsc → tools/bulk_ifsc.py
    • decode_pan_type → tools/pan_type.py
    • decode_digilocker_uri → tools/digilocker.py
    • lookup_bank → tools/bank_lookup.py
    • validate_aa_consent_artifact → tools/aa_consent.py
    • validate_epf_code → tools/epf_code.py
    • validate_esic_code → tools/esic_code.py
  • Thin Registration Wrappers: All 78 tool wrappers in registrations/ are now lightweight delegates ($\le 3$ statements) delegating execution directly to tools/ with structured error handling.
  • Worker Clamping Consolidation: Consolidated duplicate _clamp_bulk_workers implementations into a single canonical definition in src/mcp_india_stack/tools/bulk_common.py.
  • 100% Core Parity: Eliminated all tool exemptions in tests/test_parity.py. All 78 registered FastMCP tools now have direct, importable core function counterparts.
  • Golden Output Verification: Added tests/test_golden_outputs.py with 10 test suites verifying bit-for-bit identical output and schema stability across 6 diverse inputs for each extracted tool.

4. Cloud Deployment & Health Check Safeguards

  • Unauthenticated Health Probes: Public /health, /health/, and / endpoints return 200 OK ({"status": "ok", "version": "0.7.0"}) without requiring bearer token credentials or consuming rate limits.
  • Render Auto-Bind: Automatically binds to 0.0.0.0 when executing in container/PaaS environments (RENDER=true), eliminating port scan timeouts when start commands omit --host 0.0.0.0.
  • Real-Time Log Streaming: Configured line-buffering on sys.stdout and sys.stderr to prevent silent buffering in containerized deployments.

🛠️ Breaking Changes & Upgrade Notes

  1. --refresh-all CLI Flag Removed:
    Callers invoking mcp-india-stack --refresh-all will receive an exit code of 2 and an informational error message. To update dataset contents, upgrade the mcp-india-stack package via pip install --upgrade mcp-india-stack.
  2. Offline Data Resolution:
    MCP_INDIA_STACK_CACHE_DIR is no longer consulted for dataset downloads. The package strictly utilizes files packaged inside its distribution directory.
  3. Import Paths:
    Direct internal imports of the 9 extracted tools from mcp_india_stack.registrations.* should be updated to mcp_india_stack.tools.* (or from mcp_india_stack.tools import <tool>). The public FastMCP tool interface via stdio/SSE remains 100% backward-compatible.

📊 Summary of Pull Requests in v0.7.0

PR Title Core Changes
#12 Update 6.7: Remove runtime dataset refresh Offline dataset architecture, --refresh-all deprecation, .gitattributes
#13 Update 7: CI security gates & regression guards Bandit, pip-audit, lock drift check, wheel smoke, G01–G20 guards
#14 Update 5b: Extract 9 inline tools to tools package Modular tool extraction, 100% core parity, golden output verification
#15 Chore(release): Bump version to 0.7.0 Version bump across pyproject.toml, changelog, and test constants
#16 Fix(security): Allowlist newly published PYSEC-2026-4359 Exemption tracking for authlib in .pip-audit-ignore.toml
#17 Fix(deploy): Health endpoints & Render auto-bind /health route, Render 0.0.0.0 auto-bind, render.yaml health check