CHORE: Build Pipeline Modernization - Multi-Platform Support, Security Compliance & Test Infrastructure

[bewithgaurav]

Work Item / Issue Reference

AB#40501

---

Summary

This pull request makes a comprehensive overhaul to the OneBranchPipelines/build-release-package-pipeline.yml file to improve clarity, documentation, and security compliance for building and releasing the mssql-python package. The changes include detailed comments, improved parameterization, expanded platform support, and enhanced security scanning. The pipeline is now more maintainable and easier to understand, with explicit configuration for each platform and build stage.

Pipeline Structure and Documentation Improvements

• Added extensive header and inline documentation throughout the pipeline YAML, explaining platform support, build strategies, security features, and SDL compliance requirements.
• Organized pipeline sections with clear thematic dividers and comments, making the YAML much easier to read and maintain. [1] [2]

Platform and Build Configuration Enhancements

• Expanded and clarified platform build matrices for Windows, macOS, and Linux, including ARM64 and Universal2 support, with explicit configuration for each Python version and architecture.
• Updated build stages to reflect new platform configurations, including explicit dependencies for the Consolidate stage and improved artifact collection.

Security and Compliance Improvements

• Enhanced SDL (Security Development Lifecycle) configuration, enabling comprehensive security scanning (BinSkim, CredScan, PoliCheck, CodeQL, SBOM generation, etc.) and providing detailed justifications for disabled tasks.
• Improved handling of official vs. non-official builds, ensuring scheduled builds always use full SDL compliance and updating variable logic accordingly.

Variable and Resource Management

• Refactored variable templates and group imports for clarity, including effective build type logic and Azure DevOps variable group documentation.

Build Stage Details

• Added detailed comments and step-by-step breakdowns for each platform's build stage, including testing, artifact publishing, and security scanning procedures. [1] [2]

[github-actions]

📊 Code Coverage Report

🔥 Diff Coverage 100%

🎯 Overall Coverage 77%

📈 Total Lines Covered: 5065 out of 6536 📁 Project: mssql-python

---

Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.

---

📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)

mssql_python.pybind.logger_bridge.cpp: 59.2%
mssql_python.helpers.py: 66.6%
mssql_python.row.py: 67.4%
mssql_python.pybind.ddbc_bindings.cpp: 70.4%
mssql_python.pybind.connection.connection.cpp: 76.3%
mssql_python.ddbc_bindings.py: 79.6%
mssql_python.pybind.connection.connection_pool.cpp: 79.6%
mssql_python.pybind.ddbc_bindings.h: 79.7%
mssql_python.connection.py: 82.5%
mssql_python.cursor.py: 83.6%

---

🔗 Quick Links

⚙️ Build Summary	📋 Coverage Details
View Azure DevOps Build	Browse Full Coverage Report

[Copilot]

Pull Request Overview

This pull request modernizes the build pipeline with comprehensive documentation improvements, multi-platform support enhancements, and security compliance updates. However, there are critical bugs that will prevent the pipeline from running successfully.

Key Issues Found:

Critical Bugs:

1. Pipeline Dependency Mismatch: The Consolidate stage depends on Python 3.14 stages (Win_py314_x64, Win_py314_arm64, MacOS_py314) that are commented out in the configuration, which will cause the pipeline to fail waiting for non-existent stages.
2. Incorrect Variable Usage: The Consolidate stage uses ${{ parameters.oneBranchType }} instead of ${{ variables.effectiveOneBranchType }}, breaking the intended behavior where scheduled builds should always use Official mode.

Security & Best Practices:
3. Password Exposure Risk: DB_PASSWORD is passed through environment variables in shell scripts with set -x enabled, potentially exposing credentials in build logs.
4. ESRP Scanning Inconsistency: Windows builds run ESRP malware scanning unconditionally, while macOS and Linux only run it for Official builds.

Documentation Issues:
5. Extensive Comment Inconsistencies: 13 comments reference Python 3.14 support or incorrect stage counts that don't match the actual configuration.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 22 comments.

Show a summary per file

File	Description
tests/test_000_dependencies.py	Fixed module directory detection to use mssql_python.__file__ instead of string manipulation (correct fix)
OneBranchPipelines/build-release-package-pipeline.yml	Added comprehensive documentation, but introduced critical bugs with Python 3.14 stage dependencies and variable usage
OneBranchPipelines/stages/build-windows-single-stage.yml	Added Python 3.14 NuGet installation logic and ARM64 cross-compilation support; ESRP scanning runs unconditionally unlike other platforms
OneBranchPipelines/stages/build-macos-single-stage.yml	Improved documentation and Colima Docker setup; properly conditionalizes ESRP scanning
OneBranchPipelines/stages/build-linux-single-stage.yml	Added comprehensive test infrastructure with isolated pytest execution per Python version; contains duplicate commented code and password exposure risk

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The merge-base changed after approval.