Replace Azure SQL Server with PostgreSQL Flexible Server (breaking change)

[tjementum]

Summary & Motivation

Migrate the entire database layer from Azure SQL Server to PostgreSQL Flexible Server. PostgreSQL provides a lighter local development experience (smaller Docker image, faster startup), lower Azure hosting costs, and better alignment with the open-source ecosystem.

This is a clean break with no migration path from SQL Server. All existing migrations are deleted and replaced with fresh PostgreSQL-native migrations using snake_case conventions throughout.

Application changes:

• Replace the SQL Server EF Core provider with Npgsql, applying snake_case naming via EFCore.NamingConventions with explicit MigrationsHistoryTable("__ef_migrations_history") since the naming convention does not apply to the history table
• Delete all SQL Server migrations and create fresh PostgreSQL migrations using native data types (text, timestamptz, boolean, integer, jsonb) with snake_case naming, including __data_migrations_history table and fk_subscriptions_tenants_tenant_id foreign key
• Store manually serialized JSON columns (ExternalIdentities, PaymentTransactions, BillingInfo, Payload) as jsonb with .HasColumnType("jsonb") alongside HasConversion
• Rewrite DataMigrationRunner with PostgreSQL advisory locks using a deterministic SHA256-based lock key, an independent NpgsqlDataSource connection, a required per-migration Timeout (max 20 minutes), ManagesOwnTransactions opt-out for batch migrations outside execution strategy retry, and error handling on lock release
• Replace SQL Server locking patterns: FOR UPDATE in SubscriptionRepository, LINQ queries in UserRepository.GetUserSummaryAsync (with SQLite raw SQL fallback for DateTimeOffset comparison), and a single snake_case SQL statement in SessionRepository.TryRefreshAsync
• Add .OrderBy(u => u.Id) to GetUsersByEmailUnfilteredAsync since PostgreSQL does not guarantee row ordering without ORDER BY, and use SingleOrDefaultAsync for primary key lookups
• Use DefaultAzureCredential with ManagedIdentityClientId for the database token provider via UsePeriodicPasswordProvider
• Apply UseSnakeCaseNamingConvention() to SQLite test configuration and update all raw SQL across 55 test files to snake_case
• Add Plan property to Tenant to denormalize the subscription plan, restrict GetPaymentHistory to owners only
• Fix flaky Stripe webhook tests with [Collection("StripeTests")]

Infrastructure changes:

• Replace SQL Server Bicep modules with PostgreSQL Flexible Server using Entra ID-only authentication, Private Endpoint with Private DNS Zone on a dedicated private-endpoints subnet, and Ssl Mode=VerifyFull for certificate verification
• Split VNet into two subnets: container-apps (delegated to Microsoft.App/environments) and private-endpoints, since delegated subnets cannot host private endpoints
• Upgrade Container Apps from consumption-only to workload profiles with a Consumption profile, providing identical billing, higher per-app limits (4 vCPUs/8 GiB), and future dedicated compute support
• Configure pg_stat_statements for query monitoring, log_statement=mod for audit logging with diagnostic logs routed to a storage account with 90-day lifecycle retention, and wal_level=logical for logical replication support
• Chain server configuration changes after Private DNS Zone Group provisioning to prevent first-time deployment failures
• Replace Bicep dependsOn with module output references for proper implicit dependencies and --what-if visibility
• Move Entra ID admin provisioning to a post-deployment CLI step since PostgreSQL cannot provision admins with passwordAuth: 'Disabled' in a single Bicep deployment

CI/CD changes:

• Use psql with sslmode=verify-full sslrootcert=system and -v ON_ERROR_STOP=1 (without --single-transaction to support CREATE INDEX CONCURRENTLY) for migration apply, and Ssl Mode=VerifyFull for dotnet ef connections
• Open firewall once for all database permission grants instead of per-database
• Add set -e to shell scripts and IP address validation in firewall.sh
• Authenticate using the Entra admin group principal name as the PostgreSQL username with the service principal's access token
• Rename security group from "SQL Admins" to "PostgreSQL Admins" and all related GitHub variables

Documentation changes:

• Update README files, legal documents (DPA, cross-references), and developer CLI to reference PostgreSQL
• Update AI rules (api-tests.md, domain-modeling.md, database-migrations.md, backend.md, repositories.md) to reflect PostgreSQL conventions including snake_case examples, HasColumnType("jsonb") guidance, CREATE INDEX CONCURRENTLY support, and PostgreSQL ordering behavior

Downstream projects

This is a breaking change. The database provider changed from SQL Server to PostgreSQL. Delete existing SQL Server migrations and create a new initial migration using PostgreSQL conventions. There is no migration path to move data in a production system.

See the updated database-migrations rule file and the new account initial migration for reference.

Rename the Entra ID security groups in Azure from "SQL Admins" to "PostgreSQL Admins" (e.g., SQL Admins - Staging - GitHubOrganisation/GitHubRepository becomes PostgreSQL Admins - Staging - GitHubOrganisation/GitHubRepository). Then rename the following GitHub Actions variables:

• STAGING_SQL_ADMIN_OBJECT_ID to STAGING_POSTGRES_ADMIN_OBJECT_ID
• PRODUCTION_SQL_ADMIN_OBJECT_ID to PRODUCTION_POSTGRES_ADMIN_OBJECT_ID

Optional: Enable Microsoft Defender for PostgreSQL

For vulnerability assessments and security alerts, enable Microsoft Defender for Open-Source Relational Databases in your Azure subscription. This provides brute force detection, anomalous access alerts, and vulnerability scanning for PostgreSQL Flexible Server (~$15/month per server). Enable it in the Azure Portal under Microsoft Defender for Cloud > Environment settings > Defender plans, or via Azure CLI:

az security pricing create --name OpenSourceRelationalDatabases --tier Standard

Checklist

• I have added tests, or done manual regression tests
• I have updated the documentation, if necessary

[linear]

PP-983 Evaluate Postgress

[sonarqubecloud]

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

Approve Database Migration back-office database on stage

The following pending migration(s) will be applied to the database when approved:

• Initial (20250217000000_Initial)

Migration Script

CREATE TABLE IF NOT EXISTS __ef_migrations_history (
    migration_id character varying(150) NOT NULL,
    product_version character varying(32) NOT NULL,
    CONSTRAINT pk___ef_migrations_history PRIMARY KEY (migration_id)
);

START TRANSACTION;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20250217000000_Initial') THEN
    CREATE TABLE __data_migrations_history (
        migration_id text NOT NULL,
        product_version text NOT NULL,
        executed_at timestamptz NOT NULL,
        execution_time_ms bigint NOT NULL,
        summary text NOT NULL,
        CONSTRAINT pk___data_migrations_history PRIMARY KEY (migration_id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20250217000000_Initial') THEN
    INSERT INTO __ef_migrations_history (migration_id, product_version)
    VALUES ('20250217000000_Initial', '10.0.3');
    END IF;
END $EF$;
COMMIT;

[github-actions]

Approve Database Migration main database on stage

The following pending migration(s) will be applied to the database when approved:

• Initial (20260125035900_Initial)

Migration Script

CREATE TABLE IF NOT EXISTS __ef_migrations_history (
    migration_id character varying(150) NOT NULL,
    product_version character varying(32) NOT NULL,
    CONSTRAINT pk___ef_migrations_history PRIMARY KEY (migration_id)
);

START TRANSACTION;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260125035900_Initial') THEN
    CREATE TABLE __data_migrations_history (
        migration_id text NOT NULL,
        product_version text NOT NULL,
        executed_at timestamptz NOT NULL,
        execution_time_ms bigint NOT NULL,
        summary text NOT NULL,
        CONSTRAINT pk___data_migrations_history PRIMARY KEY (migration_id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260125035900_Initial') THEN
    INSERT INTO __ef_migrations_history (migration_id, product_version)
    VALUES ('20260125035900_Initial', '10.0.3');
    END IF;
END $EF$;
COMMIT;

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
8 Accepted issues

Measures
0 Security Hotspots
23.0% Coverage on New Code
2.3% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

Approve Database Migration account database on stage

The following pending migration(s) will be applied to the database when approved:

• Initial (20260303023200_Initial)

Migration Script

CREATE TABLE IF NOT EXISTS __ef_migrations_history (
    migration_id character varying(150) NOT NULL,
    product_version character varying(32) NOT NULL,
    CONSTRAINT pk___ef_migrations_history PRIMARY KEY (migration_id)
);

START TRANSACTION;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE __data_migrations_history (
        migration_id text NOT NULL,
        product_version text NOT NULL,
        executed_at timestamptz NOT NULL,
        execution_time_ms bigint NOT NULL,
        summary text NOT NULL,
        CONSTRAINT pk___data_migrations_history PRIMARY KEY (migration_id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE tenants (
        id bigint NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        deleted_at timestamptz,
        name text NOT NULL,
        state text NOT NULL,
        plan text NOT NULL,
        suspension_reason text,
        suspended_at timestamptz,
        logo jsonb NOT NULL DEFAULT '{}',
        CONSTRAINT pk_tenants PRIMARY KEY (id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE email_logins (
        id text NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        type text NOT NULL,
        email text NOT NULL,
        one_time_password_hash text NOT NULL,
        retry_count integer NOT NULL,
        resend_count integer NOT NULL,
        completed boolean NOT NULL,
        CONSTRAINT pk_email_logins PRIMARY KEY (id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE INDEX ix_email_logins_email ON email_logins (email);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE external_logins (
        id text NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        type text NOT NULL,
        provider_type text NOT NULL,
        email text,
        code_verifier text NOT NULL,
        nonce text NOT NULL,
        browser_fingerprint text NOT NULL,
        login_result text,
        CONSTRAINT pk_external_logins PRIMARY KEY (id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE users (
        tenant_id bigint NOT NULL,
        id text NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        deleted_at timestamptz,
        last_seen_at timestamptz,
        email text NOT NULL,
        external_identities jsonb NOT NULL,
        email_confirmed boolean NOT NULL,
        first_name text,
        last_name text,
        title text,
        role text NOT NULL,
        locale text NOT NULL,
        avatar jsonb NOT NULL,
        CONSTRAINT pk_users PRIMARY KEY (id),
        CONSTRAINT fk_users_tenants_tenant_id FOREIGN KEY (tenant_id) REFERENCES tenants (id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE INDEX ix_users_tenant_id ON users (tenant_id);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE UNIQUE INDEX ix_users_tenant_id_email ON users (tenant_id, email) WHERE deleted_at IS NULL;
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE sessions (
        tenant_id bigint NOT NULL,
        id text NOT NULL,
        user_id text NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        refresh_token_jti text NOT NULL,
        previous_refresh_token_jti text,
        refresh_token_version integer NOT NULL,
        login_method text NOT NULL,
        device_type text NOT NULL,
        user_agent text NOT NULL,
        ip_address text NOT NULL,
        revoked_at timestamptz,
        revoked_reason text,
        CONSTRAINT pk_sessions PRIMARY KEY (id),
        CONSTRAINT fk_sessions_tenants_tenant_id FOREIGN KEY (tenant_id) REFERENCES tenants (id),
        CONSTRAINT fk_sessions_users_user_id FOREIGN KEY (user_id) REFERENCES users (id) ON DELETE CASCADE
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE INDEX ix_sessions_tenant_id ON sessions (tenant_id);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE INDEX ix_sessions_user_id ON sessions (user_id);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE subscriptions (
        tenant_id bigint NOT NULL,
        id text NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        plan text NOT NULL,
        scheduled_plan text,
        stripe_customer_id text,
        stripe_subscription_id text,
        current_price_amount numeric(18,2),
        current_price_currency text,
        current_period_end timestamptz,
        cancel_at_period_end boolean NOT NULL,
        first_payment_failed_at timestamptz,
        cancellation_reason text,
        cancellation_feedback text,
        payment_transactions jsonb NOT NULL,
        payment_method jsonb,
        billing_info jsonb,
        CONSTRAINT pk_subscriptions PRIMARY KEY (id),
        CONSTRAINT fk_subscriptions_tenants_tenant_id FOREIGN KEY (tenant_id) REFERENCES tenants (id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE UNIQUE INDEX ix_subscriptions_tenant_id ON subscriptions (tenant_id);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE UNIQUE INDEX ix_subscriptions_stripe_customer_id ON subscriptions (stripe_customer_id) WHERE stripe_customer_id IS NOT NULL;
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE TABLE stripe_events (
        tenant_id bigint,
        id text NOT NULL,
        created_at timestamptz NOT NULL,
        modified_at timestamptz,
        event_type text NOT NULL,
        status text NOT NULL,
        processed_at timestamptz,
        stripe_customer_id text,
        stripe_subscription_id text,
        payload jsonb,
        error text,
        CONSTRAINT pk_stripe_events PRIMARY KEY (id)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE INDEX ix_stripe_events_tenant_id ON stripe_events (tenant_id);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    CREATE INDEX ix_stripe_events_stripe_customer_id_status ON stripe_events (stripe_customer_id, status);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM __ef_migrations_history WHERE "migration_id" = '20260303023200_Initial') THEN
    INSERT INTO __ef_migrations_history (migration_id, product_version)
    VALUES ('20260303023200_Initial', '10.0.3');
    END IF;
END $EF$;
COMMIT;