Testing and Verification

Testing and Verification

This page captures the normal validation flow for PumpSync changes across the iOS app and backend.

iOS Project Validation

Regenerate the Xcode project whenever files or project settings change:

xcodegen generate

List schemes and destinations:

xcodebuild -list -project PumpSync.xcodeproj
xcodebuild -showdestinations -project PumpSync.xcodeproj -scheme PumpSync

Run the main iOS test lane:

xcodebuild test -project PumpSync.xcodeproj -scheme PumpSync -destination 'platform=iOS Simulator,name=iPhone 17,OS=latest'

Keep validation output unfiltered. Raw xcodebuild may print Xcode/simulator runtime warnings that are useful for diagnosis.

iOS Test Coverage Areas

Current unit/UI coverage includes:

• Auth service behavior.
• Backend session storage.
• Cross-platform protocol 3 canonical proof-payload vectors.
• Hosted enrollment binding for the protocol domain, operation, path, installation ID, request ID, issuedAt, challenge digest, key ID, and signed-transaction-JWS digest.
• App Attest assertion and receipt-risk validation, encrypted receipt persistence and inactive-key cleanup, Secure Enclave signature validation, refresh rotation/replay handling, and active-session-family enforcement.
• Diagnostics log storage.
• Health access behavior.
• PumpSync subscription presentation.
• Initial import range logic.
• JSON codec behavior.
• Native diagnostics storage.
• Support bundle construction.
• Sync view behavior.
• UI launch coverage.

For iOS changes, include focused tests for the touched area and run the full PumpSync scheme test action when possible.

Physical Device Validation

Physical iPhone validation is separate from simulator validation. Resolve the connected destination each time:

xcodebuild -showdestinations -project PumpSync.xcodeproj -scheme PumpSync
xcrun devicectl list devices

Build/install with the resolved device ID and a fresh build output path. Do not claim launch proof unless the app was installed and opened on the actual device.

Backend Validation

Expected backend branch validation:

dotnet restore PumpSync.Backend.slnx
dotnet build PumpSync.Backend.slnx --configuration Release
dotnet test PumpSync.Backend.slnx --configuration Release
docker build -t pumpsync-backend:local .
az bicep build --file infra/bicep/main.bicep
az bicep build --file infra/bicep/main.subscription.bicep

Run the API directly (a bare dotnet run fails startup validation; the minimum environment is documented in the backend's docs/configuration.md):

PumpSync__BackendMode=SelfHosted \
PumpSync__DataSourceMode=SyntheticDemo \
PumpSync__StorageProvider=Sqlite \
SqliteStorage__DatabasePath=./pumpsync.db \
PumpSync__ServiceTokenSigningKey=$(openssl rand -base64 32) \
ASPNETCORE_URLS=http://localhost:8080 \
dotnet run --project src/PumpSync.Api/PumpSync.Api.csproj
curl http://localhost:8080/api/v1/capabilities
curl http://localhost:8080/health

Run the Docker Compose demo (the placeholder signing key must be replaced first):

cp .env.example .env
sed -i.bak "s#^PumpSync__ServiceTokenSigningKey=.*#PumpSync__ServiceTokenSigningKey=$(openssl rand -base64 32)#" .env
rm .env.bak
docker compose up --build
curl http://localhost:8080/api/v1/capabilities
curl http://localhost:8080/health

Hosted Smoke Tests

Hosted environment smoke tests should use the trusted base URL from deployment output or operator notes instead of copying live endpoint values into public docs. Include /api in the base URL unless the reverse proxy intentionally maps the API at the domain root.

Capabilities:

curl --fail <hosted-api-base-url>/v1/capabilities

Unauthenticated status should return 401:

curl --silent --output /tmp/status.json --write-out '%{http_code}' <hosted-api-base-url>/v1/status

Demo capabilities:

curl --fail https://demo.pumpsync.ericslutz.dev/api/v1/capabilities

Unauthenticated demo status should return 401:

curl --silent --output /tmp/status.json --write-out '%{http_code}' https://demo.pumpsync.ericslutz.dev/api/v1/status

Manual Verification

Automated tests do not replace real flow checks. For user-facing changes, verify the relevant path:

• PumpSync subscription purchase/restore in the correct StoreKit environment.
• Self-host URL entry and session creation.
• Background renewal without StoreKit, followed by an iOS-scheduled background sync and Azure request/log confirmation.
• Tandem credential save, validate, and delete.
• HealthKit permission request and Apple Health writes.
• Sync status, error, support, privacy, and settings copy.
• Duplicate-prevention behavior on repeated syncs.
• Synthetic demo connection with the public demo backend and demo credentials.
• Data deletion request link and installation ID support path.

Build 7 TestFlight Migration

Build 7 keeps marketing version 1.0.0 and increments only CFBundleVersion. Before adding the candidate to testers:

1. Deploy the compatible protocol 3 backend to nonproduction and confirm capabilities reports protocol 3.
2. Archive with the PumpSync Beta scheme and verify the archive carries the production App Attest entitlement while still targeting the nonproduction hosted API.
3. Test a clean hosted enrollment on a physical device. Confirm the App Attest key is created, enrolled once, reused for refresh, and reset after an explicit invalid-key response.
4. Verify hosted enrollment fails closed for a bad challenge, request-binding mismatch, and invalid local receipt. Separately exercise unavailable or non-blocking Apple risk assessment and confirm it is recorded as observe-only without preventing an otherwise valid session.
5. Verify self-hosted enrollment and renewal still use Secure Enclave proofs without an Apple Account, StoreKit entitlement, App Attest call, or receipt record.
6. Exercise foreground renewal, system-launched background renewal, and recovery after an intentionally invalidated hosted session.
7. Confirm support diagnostics report protocol 3 and the proof kind but never contain a challenge, key ID, signed transaction JWS, App Attest receipt, risk outcome, assertion, or refresh credential.
8. Verify the public privacy and deletion pages, App Store Connect privacy metadata, and TestFlight review notes match the shipped behavior before inviting testers.

Definition of Done

A change is not complete until:

• The touched repository's focused tests pass.
• Broader iOS/backend validation is run when the behavior crosses repo boundaries.
• git diff --check passes.
• Route wording preserves /api/v1 as the backend route contract.
• Wiki and source docs are updated when behavior or operational status changes.