Disaster Recovery And Resilience

Disaster Recovery And Resilience

This page documents production-like resilience thinking for the lab. The current repository is primarily a single-region learning environment, so disaster recovery is a design exercise unless explicitly implemented later.

Use this page with:

• Architecture Overview
• Network Topology
• Production Readiness Review
• Known Limitations And Design Decisions

Current Resilience Model

Area	Current lab posture
Region	Primary region driven by profile, commonly westus2
State	Remote AzureRM backend with backup action support
Identity	Primary DC and optional secondary DC in the same overall lab design
Networking	Hub-spoke model with optional VPN and simulated on-premises
Workloads	Optional load-balanced IIS, AKS, and PaaS workload services
Monitoring	Log Analytics, alerts, diagnostics, workbooks when enabled
Recovery	Destroy/recreate through Terraform; no automated regional failover

Resilience Objectives

For a lab, the objective is to prove patterns, not to guarantee uptime.

Objective	Lab target
Rebuildability	Terraform can recreate the environment from source and state
State safety	State is backed up before apply and destroy
Config clarity	Profiles explain what is enabled and what is intentionally off
Failure visibility	Monitoring and diagnostics show failures after apply
Cost control	DR features stay opt-in

Failure Scenarios

Scenario	Expected handling
Failed Terraform apply	Use state, plan output, and runbook to repair or destroy
Accidental public exposure	Stop, remove exposure, rotate credentials if needed, document incident
State corruption or loss	Restore from backup or recreate lab if state cannot be trusted
Region outage	Treat as a lab limitation unless secondary-region scenario is added
Broken private DNS	Validate zone links, private endpoint records, and VNet peering
Monitoring blind spot	Add diagnostic setting and alert coverage before calling environment ready

Optional Secondary-Region Pattern

Future production-like extension:

Component	Primary region	Secondary region option
Hub VNet	Existing hub	Secondary hub with independent address space
Shared services	Key Vault, Storage, SQL	Geo-redundant or paired-region services where supported
Workloads	Prod spoke	Secondary workload spoke
DNS	Private DNS linked to primary VNets	Link secondary VNets to same or replicated zones
Monitoring	Primary workspace	Central workspace or region-local workspace
Traffic	App Gateway or LB	Front Door or Traffic Manager pattern, if added

This should remain disabled by default because it increases cost and complexity.

State Recovery Checklist

Before restoring state:

• Confirm the failed environment and state key.
• Confirm the latest successful state backup.
• Confirm no apply/destroy is running.
• Confirm Azure resources still match the state you want to restore.
• Keep a copy of the broken state for forensic review.

After restore:

terraform init
terraform plan -var-file=environments/<profile>.tfvars

Pass condition: the plan does not propose unexpected destructive changes.

Backup And Recovery Evidence

Evidence	Why it matters
State backup action success	Proves backup ran before apply/destroy
Backend storage protection	Proves state storage is not casual public storage
Resource inventory artifact	Helps compare intended and actual Azure resources
Changelog artifact	Shows what changed and when
Cost alert	Helps detect forgotten recovery resources

DR Design Decisions To Record

Record these decisions before building a secondary-region lab:

• Which services are active-active, active-passive, or rebuild-only.
• Whether private DNS is shared or duplicated.
• Whether state remains one workspace/key per environment or splits by region.
• Which resources are intentionally not replicated due to cost.
• How traffic failover is tested.
• How teardown avoids orphaned secondary-region resources.

Next page: Known Limitations And Design Decisions