Skip to content

fix: harden Windows update reboot recovery and parameterize warpgate AMI regions - #395

Merged
mwendigg merged 1 commit into
dreadnode:mainfrom
l50:upstream/warpgate-ami-and-settings-updates
Aug 3, 2026
Merged

fix: harden Windows update reboot recovery and parameterize warpgate AMI regions#395
mwendigg merged 1 commit into
dreadnode:mainfrom
l50:upstream/warpgate-ami-and-settings-updates

Conversation

@l50

@l50 l50 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Key Changes:

  • Added connection recovery steps to survive wedged SSM channels during Windows update reboots
  • Parameterized the AWS region in warpgate AMI source images to remove hardcoded regions
  • Reduced warpgate base image volume sizes from 100GB to 50GB across all GOAD templates

Added:

  • Connection recovery after reboots - Introduced wait_for_connection tasks after both the pending-operations reboot and the update install step in settings_updates/tasks/main.yml, since wait_for_connection re-attempts a downed connection whereas until/rescue cannot ride out a wedged SSM channel on 'unreachable'
  • Explicit update completion check - Added an assert task that fails with a descriptive message (including attempt count and error) if Windows updates did not complete, ensuring a genuinely dead box is caught rather than silently passing

Changed:

  • Reboot resilience tuning - Increased reboot_timeout to 1800 and post_reboot_delay to 60, and set ignore_errors: true on the reboot and update tasks so a wedged SSM poll falls through to the new recovery and assert logic instead of failing outright (settings_updates/tasks/main.yml)
  • Region-agnostic AMI sources - Replaced hardcoded us-west-1 in the SSM parameter ARN with the ${aws_region} variable across all GOAD warpgate templates (goad-dc-base, goad-dc-base-2016, goad-mssql-base, goad-mssql-base-2016)
  • Smaller base image volumes - Lowered volume_size from 100 to 50 for all GOAD warpgate build targets to reduce cost and build footprint
  • Documentation updates - Updated settings_updates/README.md to reflect the new connection recovery and update assertion tasks in the task list

**Added:**

- Connection recovery steps after reboot and after updates using `wait_for_connection`, which re-attempts downed connections to ride out a wedged SSM channel that `until`/`rescue` cannot handle - `settings_updates/tasks/main.yml`
- Explicit assertion that fails with update attempt count and error message if Windows updates did not complete - `settings_updates/tasks/main.yml`
- Documentation of the new recovery, wait, and assert steps - `settings_updates/README.md`

**Changed:**

- Increased reboot resilience by raising `reboot_timeout` to 1800 and `post_reboot_delay` to 60, and tolerating errors on reboot and update tasks so failures fall through to the recovery and assert logic - `settings_updates/tasks/main.yml`
- Parameterized the base AMI SSM image ARN with `${aws_region}` instead of a hardcoded `us-west-1` across all warpgate templates
- Reduced AMI `volume_size` from 100 to 50 across all warpgate templates
@dreadnode-renovate-bot dreadnode-renovate-bot Bot added the area/roles Changes made to Ansible roles label Aug 3, 2026
@l50 l50 changed the title feat: add standalone agent scoring with live credential verification fix: harden Windows update reboot recovery and parameterize warpgate AMI regions Aug 3, 2026
@mwendigg
mwendigg added this pull request to the merge queue Aug 3, 2026
Merged via the queue into dreadnode:main with commit 2aea8ef Aug 3, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/roles Changes made to Ansible roles

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants