Skip to content

FEAT: Expand CI Testing to add new SQL Server Versions #308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
+263 −149
Open

FEAT: Expand CI Testing to add new SQL Server Versions #308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2ba7c29
FEAT: Enhance Testing to add new SQL Server Versions and Types
bewithgaurav Oct 29, 2025
049f0bc
added a new step itself for azure conn string
bewithgaurav Oct 29, 2025
30c452c
do the Azure SQL testing on ubuntu
bewithgaurav Oct 29, 2025
6f99ab5
skip large tests for azure sql
bewithgaurav Oct 29, 2025
30a1024
skip large tests for azure sql and fix tests
bewithgaurav Oct 29, 2025
6caebc2
test python 3.14
bewithgaurav Oct 30, 2025
405a878
python 3.14 artifacts
bewithgaurav Oct 30, 2025
e238c0d
Removed Driver param
bewithgaurav Oct 30, 2025
7f1b6c5
fix flaky test
bewithgaurav Oct 30, 2025
198e7de
commonize is_az_sql_conn
bewithgaurav Oct 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
217 changes: 173 additions & 44 deletions eng/pipelines/pr-validation-pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,32 +50,112 @@ jobs:
pool:
vmImage: 'windows-latest'

strategy:
matrix:
LocalDB:
sqlVersion: 'LocalDB'
pythonVersion: '3.13'
SQLServer2022:
sqlVersion: 'SQL2022'
pythonVersion: '3.13'
LocalDB_Python314:
sqlVersion: 'LocalDB'
pythonVersion: '3.14'

steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.13'
versionSpec: '$(pythonVersion)'
addToPath: true
githubToken: $(GITHUB_TOKEN)
displayName: 'Use Python 3.13'
displayName: 'Use Python $(pythonVersion)'

- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
displayName: 'Install dependencies'

# Start LocalDB instance
# Start LocalDB instance (for LocalDB matrix)
- powershell: |
sqllocaldb create MSSQLLocalDB
sqllocaldb start MSSQLLocalDB
displayName: 'Start LocalDB instance'
condition: eq(variables['sqlVersion'], 'LocalDB')

# Create database and user
# Create database and user for LocalDB
- powershell: |
sqlcmd -S "(localdb)\MSSQLLocalDB" -Q "CREATE DATABASE TestDB"
sqlcmd -S "(localdb)\MSSQLLocalDB" -Q "CREATE LOGIN testuser WITH PASSWORD = '$(DB_PASSWORD)'"
sqlcmd -S "(localdb)\MSSQLLocalDB" -d TestDB -Q "CREATE USER testuser FOR LOGIN testuser"
sqlcmd -S "(localdb)\MSSQLLocalDB" -d TestDB -Q "ALTER ROLE db_owner ADD MEMBER testuser"
displayName: 'Setup database and user'
displayName: 'Setup database and user for LocalDB'
condition: eq(variables['sqlVersion'], 'LocalDB')
env:
DB_PASSWORD: $(DB_PASSWORD)

# Install SQL Server 2022 (for SQL2022 matrix)
- powershell: |
Write-Host "Downloading SQL Server 2022 Express..."
# Download SQL Server 2022 Express installer
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -Uri "https://go.microsoft.com/fwlink/p/?linkid=2216019" -OutFile "SQL2022-SSEI-Expr.exe"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A maintenance note. When a new version of SQL, esp CU is released, this old URL typically becomes ineffective.
We have 1ES images which leverage similar script for download and install of SQL. There is a central team which updates these URLs regularly.

This comment is FYI. You can't do anything about it. But if one fine day, the CI starts failing then you know what to do 😄


Write-Host "Installing SQL Server 2022 Express..."
# Install SQL Server 2022 Express with basic features
Start-Process -FilePath "SQL2022-SSEI-Expr.exe" -ArgumentList "/Action=Download","/MediaPath=$env:TEMP","/MediaType=Core","/Quiet" -Wait

# Find the downloaded setup file
$setupFile = Get-ChildItem -Path $env:TEMP -Filter "SQLEXPR_x64_ENU.exe" -Recurse | Select-Object -First 1

if ($setupFile) {
Write-Host "Extracting SQL Server setup files..."
Start-Process -FilePath $setupFile.FullName -ArgumentList "/x:$env:TEMP\SQLSetup","/u" -Wait

Write-Host "Running SQL Server setup..."
Start-Process -FilePath "$env:TEMP\SQLSetup\setup.exe" -ArgumentList "/Q","/ACTION=Install","/FEATURES=SQLEngine","/INSTANCENAME=MSSQLSERVER","/SQLSVCACCOUNT=`"NT AUTHORITY\SYSTEM`"","/SQLSYSADMINACCOUNTS=`"BUILTIN\Administrators`"","/TCPENABLED=1","/SECURITYMODE=SQL","/SAPWD=$(DB_PASSWORD)","/IACCEPTSQLSERVERLICENSETERMS" -Wait
} else {
Write-Error "Failed to download SQL Server setup file"
exit 1
}

Write-Host "SQL Server 2022 installation completed"
displayName: 'Install SQL Server 2022 Express'
condition: eq(variables['sqlVersion'], 'SQL2022')
env:
DB_PASSWORD: $(DB_PASSWORD)

# Create database for SQL Server 2022
- powershell: |
# Wait for SQL Server to start
$maxAttempts = 30
$attempt = 0
$connected = $false

Write-Host "Waiting for SQL Server 2022 to start..."
while (-not $connected -and $attempt -lt $maxAttempts) {
try {
sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -Q "SELECT 1" -C
$connected = $true
Write-Host "SQL Server is ready!"
} catch {
$attempt++
Write-Host "Waiting... ($attempt/$maxAttempts)"
Start-Sleep -Seconds 2
}
}

if (-not $connected) {
Write-Error "Failed to connect to SQL Server after $maxAttempts attempts"
exit 1
}

# Create database and user
sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -Q "CREATE DATABASE TestDB" -C
sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -Q "CREATE LOGIN testuser WITH PASSWORD = '$(DB_PASSWORD)'" -C
sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -d TestDB -Q "CREATE USER testuser FOR LOGIN testuser" -C
sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -d TestDB -Q "ALTER ROLE db_owner ADD MEMBER testuser" -C
displayName: 'Setup database and user for SQL Server 2022'
condition: eq(variables['sqlVersion'], 'SQL2022')
env:
DB_PASSWORD: $(DB_PASSWORD)

Expand All @@ -84,31 +164,48 @@ jobs:
build.bat x64
displayName: 'Build .pyd file'

# Run tests for LocalDB
- script: |
python -m pytest -v --junitxml=test-results.xml --cov=. --cov-report=xml --capture=tee-sys --cache-clear
displayName: 'Run tests with coverage'
python -m pytest -v --junitxml=test-results-localdb.xml --cov=. --cov-report=xml:coverage-localdb.xml --capture=tee-sys --cache-clear
displayName: 'Run tests with coverage on LocalDB'
condition: eq(variables['sqlVersion'], 'LocalDB')
env:
DB_CONNECTION_STRING: 'Server=(localdb)\MSSQLLocalDB;Database=TestDB;Uid=testuser;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes'

- task: PublishBuildArtifacts@1
# Run tests for SQL Server 2022
- script: |
python -m pytest -v --junitxml=test-results-sql2022.xml --cov=. --cov-report=xml:coverage-sql2022.xml --capture=tee-sys --cache-clear
displayName: 'Run tests with coverage on SQL Server 2022'
condition: eq(variables['sqlVersion'], 'SQL2022')
env:
DB_CONNECTION_STRING: 'Server=localhost;Database=TestDB;Uid=testuser;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes'

- task: CopyFiles@2
inputs:
PathtoPublish: 'mssql_python/ddbc_bindings.cp313-amd64.pyd'
ArtifactName: 'ddbc_bindings'
publishLocation: 'Container'
displayName: 'Publish pyd file as artifact'
SourceFolder: 'mssql_python'
Contents: 'ddbc_bindings.cp*-amd64.pyd'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
displayName: 'Copy pyd file to staging'

- task: CopyFiles@2
inputs:
SourceFolder: 'mssql_python'
Contents: 'ddbc_bindings.cp*-amd64.pdb'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
displayName: 'Copy pdb file to staging'

- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: 'mssql_python/ddbc_bindings.cp313-amd64.pdb'
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'ddbc_bindings'
publishLocation: 'Container'
displayName: 'Publish pdb file as artifact'
displayName: 'Publish build artifacts'

- task: PublishTestResults@2
condition: succeededOrFailed()
inputs:
testResultsFiles: '**/test-results.xml'
testRunTitle: 'Publish test results'
testResultsFiles: '**/test-results-*.xml'
testRunTitle: 'Publish test results for Windows $(sqlVersion)'

# - task: PublishCodeCoverageResults@1
# inputs:
Expand Down Expand Up @@ -190,7 +287,7 @@ jobs:
python -m pytest -v --junitxml=test-results.xml --cov=. --cov-report=xml --capture=tee-sys --cache-clear
displayName: 'Run pytest with coverage'
env:
DB_CONNECTION_STRING: 'Driver=ODBC Driver 18 for SQL Server;Server=tcp:127.0.0.1,1433;Database=master;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes'
DB_CONNECTION_STRING: 'Server=tcp:127.0.0.1,1433;Database=master;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes'
DB_PASSWORD: $(DB_PASSWORD)

- task: PublishTestResults@2
Expand All @@ -209,9 +306,23 @@ jobs:
Ubuntu:
dockerImage: 'ubuntu:22.04'
distroName: 'Ubuntu'
sqlServerImage: 'mcr.microsoft.com/mssql/server:2022-latest'
useAzureSQL: 'false'
Ubuntu_SQL2025:
dockerImage: 'ubuntu:22.04'
distroName: 'Ubuntu-SQL2025'
sqlServerImage: 'mcr.microsoft.com/mssql/server:2025-latest'
useAzureSQL: 'false'
Ubuntu_AzureSQL:
dockerImage: 'ubuntu:22.04'
distroName: 'Ubuntu-AzureSQL'
sqlServerImage: ''
useAzureSQL: 'true'
Debian:
dockerImage: 'debian:12'
distroName: 'Debian'
sqlServerImage: 'mcr.microsoft.com/mssql/server:2022-latest'
useAzureSQL: 'false'

steps:
- script: |
Expand All @@ -230,7 +341,7 @@ jobs:
-e ACCEPT_EULA=Y \
-e MSSQL_SA_PASSWORD="$(DB_PASSWORD)" \
-p 1433:1433 \
mcr.microsoft.com/mssql/server:2022-latest
$(sqlServerImage)

# Wait for SQL Server to be ready
echo "Waiting for SQL Server to start..."
Expand All @@ -256,6 +367,7 @@ jobs:
-P "$(DB_PASSWORD)" \
-C -Q "CREATE DATABASE TestDB"
displayName: 'Start SQL Server container for $(distroName)'
condition: eq(variables['useAzureSQL'], 'false')
env:
DB_PASSWORD: $(DB_PASSWORD)

Expand Down Expand Up @@ -354,20 +466,35 @@ jobs:

- script: |
# Run tests in the container
# Get SQL Server container IP
SQLSERVER_IP=$(docker inspect sqlserver-$(distroName) --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}')
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-$(distroName) bash -c "
source /opt/venv/bin/activate
echo 'Build successful, running tests now on $(distroName)'
echo 'Using connection string: Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
python -m pytest -v --junitxml=test-results-$(distroName).xml --cov=. --cov-report=xml:coverage-$(distroName).xml --capture=tee-sys --cache-clear
"
if [ "$(useAzureSQL)" = "true" ]; then
# Azure SQL Database testing
echo "Testing against Azure SQL Database"

docker exec \
-e DB_CONNECTION_STRING="$(AZURE_CONNECTION_STRING)" \
test-container-$(distroName) bash -c "
source /opt/venv/bin/activate
echo 'Build successful, running tests now on $(distroName) with Azure SQL'
echo 'Using Azure SQL connection string'
python -m pytest -v --junitxml=test-results-$(distroName).xml --cov=. --cov-report=xml:coverage-$(distroName).xml --capture=tee-sys --cache-clear
"
else
# Local SQL Server testing
SQLSERVER_IP=$(docker inspect sqlserver-$(distroName) --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}')
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-$(distroName) bash -c "
source /opt/venv/bin/activate
echo 'Build successful, running tests now on $(distroName)'
echo 'Using connection string: Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
python -m pytest -v --junitxml=test-results-$(distroName).xml --cov=. --cov-report=xml:coverage-$(distroName).xml --capture=tee-sys --cache-clear
"
fi
displayName: 'Run pytest with coverage in $(distroName) container'
condition: or(eq(variables['useAzureSQL'], 'false'), and(eq(variables['useAzureSQL'], 'true'), ne(variables['AZURE_CONNECTION_STRING'], '')))
env:
DB_PASSWORD: $(DB_PASSWORD)

Expand All @@ -382,8 +509,10 @@ jobs:
# Clean up containers
docker stop test-container-$(distroName) || true
docker rm test-container-$(distroName) || true
docker stop sqlserver-$(distroName) || true
docker rm sqlserver-$(distroName) || true
if [ "$(useAzureSQL)" = "false" ]; then
docker stop sqlserver-$(distroName) || true
docker rm sqlserver-$(distroName) || true
fi
displayName: 'Clean up $(distroName) containers'
condition: always()

Expand Down Expand Up @@ -570,13 +699,13 @@ jobs:
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-$(distroName)-$(archName) bash -c "
source /opt/venv/bin/activate
echo 'Build successful, running tests now on $(distroName) ARM64'
echo 'Architecture:' \$(uname -m)
echo 'Using connection string: Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
echo 'Using connection string: Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
python main.py
python -m pytest -v --junitxml=test-results-$(distroName)-$(archName).xml --cov=. --cov-report=xml:coverage-$(distroName)-$(archName).xml --capture=tee-sys --cache-clear
"
Expand Down Expand Up @@ -778,12 +907,12 @@ jobs:
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-rhel9 bash -c "
source myvenv/bin/activate
echo 'Build successful, running tests now on RHEL 9'
echo 'Using connection string: Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
echo 'Using connection string: Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
python main.py
python -m pytest -v --junitxml=test-results-rhel9.xml --cov=. --cov-report=xml:coverage-rhel9.xml --capture=tee-sys --cache-clear
"
Expand Down Expand Up @@ -997,13 +1126,13 @@ jobs:
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-rhel9-arm64 bash -c "
source myvenv/bin/activate
echo 'Build successful, running tests now on RHEL 9 ARM64'
echo 'Architecture:' \$(uname -m)
echo 'Using connection string: Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
echo 'Using connection string: Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
python -m pytest -v --junitxml=test-results-rhel9-arm64.xml --cov=. --cov-report=xml:coverage-rhel9-arm64.xml --capture=tee-sys --cache-clear
"
displayName: 'Run pytest with coverage in RHEL 9 ARM64 container'
Expand Down Expand Up @@ -1225,13 +1354,13 @@ jobs:
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-alpine bash -c "
echo 'Build successful, running tests now on Alpine x86_64'
echo 'Architecture:' \$(uname -m)
echo 'Alpine version:' \$(cat /etc/alpine-release)
echo 'Using connection string: Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
echo 'Using connection string: Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'

# Activate virtual environment
source /workspace/venv/bin/activate
Expand Down Expand Up @@ -1467,13 +1596,13 @@ jobs:
echo "SQL Server IP: $SQLSERVER_IP"

docker exec \
-e DB_CONNECTION_STRING="Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \
-e DB_PASSWORD="$(DB_PASSWORD)" \
test-container-alpine-arm64 bash -c "
echo 'Build successful, running tests now on Alpine ARM64'
echo 'Architecture:' \$(uname -m)
echo 'Alpine version:' \$(cat /etc/alpine-release)
echo 'Using connection string: Driver=ODBC Driver 18 for SQL Server;Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'
echo 'Using connection string: Server=$SQLSERVER_IP;Database=TestDB;Uid=SA;Pwd=***;TrustServerCertificate=yes'

# Activate virtual environment
source /workspace/venv/bin/activate
Expand Down Expand Up @@ -1574,7 +1703,7 @@ jobs:
lcov_cobertura total.info --output unified-coverage/coverage.xml
displayName: 'Generate unified coverage (Python + C++)'
env:
DB_CONNECTION_STRING: 'Driver=ODBC Driver 18 for SQL Server;Server=tcp:127.0.0.1,1433;Database=master;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes'
DB_CONNECTION_STRING: 'Server=tcp:127.0.0.1,1433;Database=master;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes'
DB_PASSWORD: $(DB_PASSWORD)

- task: PublishTestResults@2
Expand Down
Loading
Loading