v0.0.12

v0.0.12 fixes the restore defect behind caltechauthors issue #185 and adds a health check for RDM's log indices. A restore with include_global_state:false carries neither index templates nor data-stream definitions, so the app auto-created plain, text-mapped audit-log and job-logs indices and the admin audit-logs page silently stopped updating (HTTP 200, _shards.failed:1).

Three new forms under clasm rdm-backup-and-restore <slug> (RDM menu items 6, 7 and 8), each with -text|-json and -wait (default 5m, 0 checks once). rdm-index-health is a read-only check of the audit log and job logs: exit 0 healthy, 1 a problem (missing template, plain or orphaned log index, text mappings, failed shards, a plain index that a data-stream template claims). install-index-templates installs the templates the app would install, only the missing ones, through the app's own routine; it never overwrites, and a run with nothing missing exits 0 (DR-0199). convert-log-indices turns orphaned and plain log indices back into real data streams, copying first; it needs --confirm on a production instance (DR-0180). Copies are named clasm-fix-* and clasm never deletes them.

Restore OpenSearch (menu and command line) now ends with the install, the conversion and the health check, after the verified restore and the snapshot cleanup, with no second confirmation (DR-0199). The check's verdict is the restore's: a restore whose health check fails exits 1. Log indices are found by what they are, not by the alias, which a live restore of a pre-repair snapshot showed to matter. Verified live on a test instance.

Snapshots taken from a real data stream restore as a real data stream; older snapshots hold the orphan form, so the conversion stays.

Full Changelog: v0.0.11...v0.0.12

v0.0.11

v0.0.11 makes the CaltechAUTHORS restore scriptable and safe to run on a test instance, tags everything clasm creates, adds a report and a command for tagging what was there before, and adds command-line forms to start and stop an instance.

Restore role and instance profile. A test instance that restores from production's backups should not be able to write to the backup buckets, so clasm can now build and tear down that setup from the command line. clasm iam create-role-from-template has a new template, "RDM Restore Source (read-only)": SSM, read of one source prefix in one or two backup buckets, and an explicit Deny of every write and delete there (DR-0184). create-instance-profile (also a new IAM menu item, number 7, so the items after it move down by one) wraps a role in a profile. clasm compute associate-replace-iam-instance-profile <instance> <profile> swaps a running instance's profile in place, no reboot, and is a no-op when the instance already has it; it refuses a profile that is not SSM-capable (DR-0185). remove-role-from-instance-profile, delete-instance-profile and delete-role complete the teardown on the shared destructive gate: a dry run by default and --confirm <name> to run (DR-0186, DR-0188).

Restore location guard. A restored RDM database carries its file locations (CaltechAUTHORS: s3://caltechauthors), so an instance that can write to that bucket would write to production's content after the restore. Restore SQL Backup now reads the backup's files_location rows before replacing anything (a dry run streams the dump through the instance and writes nothing), asks IAM's policy simulator whether the instance's role or the access key in /Sites/.env can write to each S3 bucket, and refuses unless each writable bucket is named in its own --confirm, beside the instance's: --confirm <instance> --confirm <bucket> (DR-0187). An instance that cannot write to any of them is never blocked. What cannot be told is treated as writable, and the simulator does not see bucket policies. It needs iam:SimulatePrincipalPolicy, iam:GetAccessKeyLastUsed and iam:GetUser. The menu flow warns and asks to continue. One behaviour change: --confirm "" is now a usage error instead of a silent dry run.

Tagging. clasm now adds the Origin tag (the key and DLD-owned value in origin_tag in ~/.clasm) to everything it creates: IAM roles, policies and instance profiles; EC2 instances with their volumes and network interfaces, launch templates and what they launch, AMIs with their snapshots, key pairs and the temporary cloud-init instance; and S3 buckets (DR-0189, DR-0190, DR-0191). It only does so when origin_tag.dld_value is set; with the default empty value nothing is added. A tag the request already carries keeps its own value. S3 objects and new launch template versions are not tagged, and nothing that already existed is changed.

Finding and tagging what was there before. Tag Management now covers eleven kinds: EBS volumes, EBS snapshots and network interfaces join the original eight (ebs-volume, ebs-snapshot, network-interface). clasm tag-management show-untagged [-text|-json|-jsonl] [<kind>...] lists the resources that lack the Origin tag, then those that have it with another value; it is read-only, takes kinds to narrow it, and leaves out AWS service-linked IAM roles. clasm tag-management tag-resource [--replace] <kind> <name-or-id> --tag key=value ... adds tags to one resource from a script: it adds what is missing, treats the same value as done so a loop can be re-run, and refuses a different value, changing nothing, unless --replace is given. It rejects aws: keys and an Environment value other than production, development or test (DR-0192, DR-0193). show-untagged is also item 3 of the Tag Management menu. The command line is still experimental: names and JSON keys may change.

Starting and stopping an instance. clasm compute start-ec2-instance [-no-wait] <instance> and clasm compute stop-ec2-instance [-no-wait] [--confirm <instance>] <instance> are the command-line forms of the two Compute menu items (DR-0194, DR-0195). Both read the instance's live state from AWS rather than the listing clasm loaded at startup, and both are safe to repeat: starting a running instance, or stopping a stopped one, sends nothing and succeeds, and an instance already starting or stopping is waited out. A state that cannot take the request (a stopping instance cannot be started; a pending, shutting-down or terminated one cannot be stopped) is exit 1 with nothing sent. They wait for the end state, up to five minutes, unless -no-wait is given, which returns as soon as AWS accepts the request and says the state it is in (pending or stopping). Stopping an instance whose live Environment tag is exactly production needs --confirm <instance-id-or-name>: without it the form prints the plan, changes nothing and exits 0, and a terminal is asked to type the name. --confirm is accepted on any instance and checked against it, before any AWS call. With -json both print one object (instance_id, name, region, previous_state, state, changed, plus the connection details for a start and declined for a stop). An unreachable AWS is exit 69. Terminate stays menu-only.

Note on verification. Verified against real AWS (us-west-2, 2026-10-16 and 2026-10-19): the restore role and its policy with the policy simulator; the profile swap and its repeat; the location guard on a real 1002.9 MiB production dump, as a dry run, as a confirmed restore with the read-only role, and refusing and exiting 2 in the writable cases; Restore OpenSearch of a 6.4 GiB snapshot, and then the full Archive OpenSearch Snapshot workflow on that restored instance into a scratch prefix (exit 0 in 246 s, 3,219 objects and 6,916,570,422 bytes, confirmed by clasm and independently by the AWS CLI, with the repository directory still owned by the service user and no root-owned file: the restore-then-archive direction DR-0175 fixed); tagging on launch templates, on launches from templates (including templates with no tag) and on creation of every EC2 and IAM kind; the whole-account show-untagged and tag-resource on ten kinds of throwaway resource, including the refusal and --replace, and tag-resource on a real S3 bucket, whose three existing tags all survived the merge; start-ec2-instance and stop-ec2-instance (44 checks on a test instance: the waits, the repeats, -no-wait each way, waiting out a pending and a stopping instance, a wrong and a right --confirm); and the teardown forms. The full-screen flows were also driven at a terminal against AWS: Create Instance Profile (the profile came out tagged and SSM-capable; a name already in use was refused by AWS with nothing created), Show untagged resources over all eleven kinds, Manage tags on the three new kinds (a tag added to and removed from an EBS volume, and the menu editor accepting any Environment value, as DR-0193 records), and the Restore SQL Backup menu as far as the file-location guard and the type-the-name prompt, which a blank answer cancelled before anything was downloaded. Not verified: the stop of a production instance (its --confirm gate is unit-tested only; no production instance was touched); the guard's warn-and-ask prompt, which appears only when the instance can write to a bucket the backup names (unit-tested only); exit 69; and two small checks not run by hand, declining the Create Instance Profile confirm and the CLI refusal of an Environment value outside production, development and test (both unit-tested). Rough edges found while testing are listed in TODO.md and are not part of this release.

Full Changelog: v0.0.10...v0.0.11

v0.0.10

v0.0.10 takes clasm's command line from two scriptable actions to 23 of its 58 menu actions, makes the destructive ones safe to script, and fixes several things found by running clasm against real instances.

Command-line forms. Every domain now has a slug, and these actions run non-interactively: Compute (show-instances, show-amis, show-launch-templates, the instance, AMI and launch template detail views, show-export-cloud-init-for-an-instance-or-ami, create-launch-template-from-cloud-init-yaml, create-ec2-instance-from-launch-template), Key Management (show-key-pairs), IAM (the roles, instance profiles and policies listings and the role and instance profile details), S3 (show-buckets), Tag Management (show-all-tags ), Configuration (show-current-config), and RDM Backup & Restore (Generate SQL Backup, Restore SQL Backup and Restore OpenSearch Snapshot, beside the two archive forms from v0.0.9). Listings take -text (the default), -json or -jsonl; details take -text or -json. The command line is experimental: names and JSON keys may still change. The two destructive Restore forms share one gate: a dry run by default that prints the plan and changes nothing, and --confirm <instance-id-or-name> to run it without a prompt (DR-0180). The editors and Save in Configure stay interactive.

Launch templates. Instances launched from a template made by "Create launch template from cloud-init YAML" now carry a templateName tag with the template's name (issue #1). A new instance's output also shows the aws ssm start-session command beside the ssh example, with or without a public address.

Reading an AMI's cloud-init never worked before this release, for two independent reasons, both fixed. The temporary instance launched into the VPC default security group, which here has no outbound rules, so its SSM agent never registered; it now takes a group (cloud_init_extraction_security_groups, one per region, with a Configure editor that lists each region's groups and marks which allow outbound HTTPS, or -security-group on the command line) and refuses a group with no outbound port 443 before launching anything. And it read the new instance's empty user-data instead of the AMI source's gzipped one. cloud-init finishing "degraded" (warnings only) is no longer reported as an error, and a real error now shows cloud-init's own text.

Restore SQL Backup loads about twenty times faster (synchronous_commit=off for the load session only: 5.2 GB in 9 minutes 27 seconds, against about five hours), and removes its two scratch files, about 6.2 GB for production's database, once the table count is verified; a failed restore keeps them. Restore OpenSearch Snapshot now moves stale repository files aside, handles data streams, stops on an empty source, and deletes live indices only after the source is verified. Generate SQL Backup needs Docker rather than the AWS CLI and shows pg_dump's own error. Archive OpenSearch repairs ownership below the top of its repository directory, and only where that directory looks like a snapshot repository.

Backup directories. Generate SQL Backup never changes an existing directory's owner, group or mode: it creates a missing one as ubuntu:www-data 0770 and hands only the new dump to the service user, in the directory's own group, mode 0664 (DR-0183). The cron that writes the nightly dumps on the production RDM hosts runs as a member of www-data, not as ubuntu, so re-owning the directory would lock it out.

The documentation site is now built in GitHub Actions from docs/ with the shared caltechlibrary/workflows (the first run published successfully). make website and the committed HTML are gone; make, make release and release.bash do not depend on GitHub Actions.

Note on verification. Verified against real AWS: the two archive forms, both Restore forms and the shared gate on caltechauthors-test-v13 (a confirmed Restore SQL of production's dump, scratch files removed); every read-only form; the create-launch-template and create-instance forms; reading cloud-init from both AMIs in this account; Sync and Modify Size against a scratch launch template; the OpenSearch ownership repair; the Generate SQL Backup directory behaviour (an existing root:www-data 0770 directory left unchanged and the new dump handed over as ubuntu:www-data 0664, on caltechauthors-test-v13); the documentation workflow on GitHub. Not verified: the interactive pickers (their prompts were exercised in accessible mode, not the full-screen terminal); the AWS CLI install added to four cloud-init files, which has been checked statically but not by a boot.

Full Changelog: v0.0.9...v0.0.10

v0.0.9

clasm can now run two RDM backup archive actions directly from the command line, without the interactive menu: clasm rdm-backup-and-restore archive-sql-backups-to-s3 <instance> <directory> <bucket> <trim-days-or-""> and the equivalent for Archive OpenSearch Snapshot to S3. This is meant for a trusted, repeatable operation moved into a script or cron job -- the same preflight checks the interactive menu runs (AWS CLI availability, bucket access) still run, only the confirmation prompt is skipped, and an unrecognized domain/action name or the wrong number of arguments fails immediately with a usage error rather than hanging or guessing.

The command line is a small domain-specific language mirroring the menu itself: clasm <domain> drops straight into that domain's own menu, clasm <domain> <action> drops straight into that one action's own prompts, and adding the action's own arguments after that runs it non-interactively instead. Backing out of either deep-link behaves exactly like normal menu navigation -- there is no dead end, and no separate command shape to learn for scripted versus interactive use.

After a successful interactive run of either archive action, clasm now prints the exact non-interactive command that reproduces it, shell-quoted and ready to paste into a crontab entry -- the natural way to promote a run from 'I did this by hand and it worked' to 'this runs on a schedule now' without hand-transcribing arguments.

See user_manual.md, 'Non-interactive (CLI) Usage' for the full syntax.

Note on verification: verified against real AWS -- the incomplete-argument usage error, both deep-link forms, and a full non-interactive Archive OpenSearch Snapshot to S3 run against CaltechAUTHORS production (3,196 objects, ~6.9GB synced), all confirmed live. Archive SQL Backups to S3's own non-interactive form was not separately exercised live; it shares the same code path.

Full Changelog: v0.0.8...v0.0.9

v0.0.8

Archive SQL Backup to S3 now copies every backup in the directory and verifies each copy, trimming only the ones old enough to delete. Previously one age-filtered set drove the upload, the verification and the deletion alike, so a dump too recent to trim was never copied at all -- a directory with nothing old enough reported 'No files match the age threshold. Nothing to do.' having archived nothing, which is exactly the case where the newest dump most needs to be in S3. A HeadObject pre-pass skips what is already archived, a blank retention answer means copy but delete nothing, and a per-file upload failure now warns and continues rather than aborting the run.

OpenSearch snapshots now include Invenio's audit logs and job logs. The allowlist previously carried only the data-stream form of the audit log, so on an instance restored from a snapshot -- which is every instance after a migration -- the historical audit log was backed up and every audit record written since go-live was not. Both index forms are now listed, along with job logs in both forms. Note the restore-side consequence: a restore now deletes and replaces a target's matching audit-log and job-log indices where it previously left them alone.

Restoring an OpenSearch snapshot no longer leaves the target unable to archive. The S3 sync runs as root on the host, so it wrote the entire snapshot repository root-owned, while OpenSearch runs as uid 1000 inside its container and could only write the top-level directory. Nothing restored the ownership, and because repository registration only ever verifies a write into that top-level directory, both the registration and the subsequent snapshot request reported success -- the failure surfaced only later, as every shard failing to write. The restore now sets ownership across the whole repository after syncing and before registering, and once a restore is verified it deletes the local snapshot through the OpenSearch API and deregisters the repository, leaving the instance ready to archive instead of accumulating stale snapshots. Nothing destructive runs before that verification succeeds. Relatedly, a snapshot that fails after being accepted is no longer reported as a bare 404 for a snapshot clasm created seconds earlier: it now says that creation failed after acceptance and points at the search container's own log, where the real cause is, and a missing snapshot is tolerated briefly first in case a busy cluster simply has not caught up.

Note on verification: the SQL archive change and the log-index change were both verified against real AWS. The OpenSearch repository-ownership and cleanup changes are unit-tested but NOT yet verified against real AWS -- the instance that exhibited the bug was repaired by hand and can no longer reproduce it, so live verification is deferred to the next real migration.

Full Changelog: v0.0.7...v0.0.8

v0.0.7

Fixes a recurring 'InvalidUserData.Malformed: User data is limited to 16384 bytes' error on large cloud-init files: gzip compression now uses maximum compression, and user-data is checked against AWS's limit before any API call, failing with a clear, actionable clasm-side error (stating the compressed size and the overage) instead of AWS's opaque 400. Fixes two related bugs in the cloud-init file picker: a bare 'q' now cancels (matching every other picker's convention, instead of being treated as a literal filename), and cancelling anywhere in Sync Cloud-Init YAML to a Launch Template now correctly returns to the Compute menu instead of exiting clasm entirely. Adds a new action, Modify Launch Template's Instance Type / EBS Root Volume Size, creating a new launch template version with both overrides in one step (never auto-promoted, matching Sync); if the newly chosen instance type's CPU architecture doesn't match the template's current AMI (e.g. switching a Graviton/arm64 template to an x86_64 instance type), it prompts for a compatible replacement AMI automatically. Finally, the connection info shown after launching or starting an instance now includes a ready-to-use '-i ' key reference (guessed from where clasm itself saves a key pair's private key, shown only when that file actually exists) and guesses the SSH login username ('ubuntu' for this tool's own curated/official Ubuntu AMIs, 'ec2-user' otherwise) instead of always hardcoding 'ec2-user'. Adds a new RDM Backup & Restore domain (a 7th top-level menu) consolidating Invenio RDM's own backup/restore lifecycle: Generate SQL Backup triggers a pg_dump directly via SSM with no pre-installed script required; Archive SQL Backup to S3 (relocated here from Compute) and a new Archive OpenSearch Snapshot to S3 (via OpenSearch's native snapshot API, with app-managed S3-side cleanup) upload dated backups; and two new restore actions -- Restore SQL Backup from S3 and Restore OpenSearch Snapshot from S3 (the latter safely deleting any conflicting indices before restoring) -- bring either kind of backup back onto a target instance, both real-AWS-verified end to end against real production data. Long OpenSearch snapshot/restore waits and any other long-running remote command now report visible progress instead of leaving the terminal looking hung, and a real reliability bug is fixed where AWS's own SSM execution silently capped any long-running remote command at one hour regardless of clasm's own configured timeout -- this could truncate a legitimate multi-hour restore without any clear error. Several smaller fixes round out this release: Generate SQL Backup no longer auto-chains into Archive SQL Backup; Delete Role's error message and two new IAM actions (Remove Role from Instance Profile, Delete Instance Profile) now give the actual remedy when a role can't be deleted because it's still attached to an instance profile; Associate/replace IAM instance profile handles an already-existing profile name gracefully instead of trapping the operator in a confusing loop; newly created instances/AMIs now tag correctly per the fleet's lowercase 'project' tag convention; and Configure clasm edits (backup directories, Postgres container overrides, the Origin tag) take effect immediately instead of requiring a restart.

Full Changelog: v0.0.6...v0.0.7

v0.0.6

Fixes a recurring 'InvalidUserData.Malformed: User data is limited to 16384 bytes' error on large cloud-init files: gzip compression now uses maximum compression, and user-data is checked against AWS's limit before any API call, failing with a clear, actionable clasm-side error (stating the compressed size and the overage) instead of AWS's opaque 400. Fixes two related bugs in the cloud-init file picker: a bare 'q' now cancels (matching every other picker's convention, instead of being treated as a literal filename), and cancelling anywhere in Sync Cloud-Init YAML to a Launch Template now correctly returns to the Compute menu instead of exiting clasm entirely. Adds a new action, Modify Launch Template's Instance Type / EBS Root Volume Size, creating a new launch template version with both overrides in one step (never auto-promoted, matching Sync); if the newly chosen instance type's CPU architecture doesn't match the template's current AMI (e.g. switching a Graviton/arm64 template to an x86_64 instance type), it prompts for a compatible replacement AMI automatically. Finally, the connection info shown after launching or starting an instance now includes a ready-to-use '-i ' key reference (guessed from where clasm itself saves a key pair's private key, shown only when that file actually exists) and guesses the SSH login username ('ubuntu' for this tool's own curated/official Ubuntu AMIs, 'ec2-user' otherwise) instead of always hardcoding 'ec2-user'.

Full Changelog: v0.0.5...v0.0.6

v0.0.5

Adds a fifth domain, IAM: browse Roles/Instance Profiles/Policies (each showing a config-driven 'Origin' tag's value, or '(unset)'), a read-only guard on permission-changing actions for anything not recognized as DLD-owned (tagging itself is always allowed), a detail view (trust policy, attached policies, tags, SSM-capability, cross-references), five curated per-use-case role/policy creation templates (Static Website, RDM Repository, Bridge Service, Patron-Facing Service, Data Processing), and CRUD completion for DLD-owned roles (Delete Role, Attach/Detach Policy). Also adds: SSM-capable instance profile enforcement at launch plus associate/replace for already-running instances; automatic gzip compression of user-data, fixing AWS's 16384-byte limit on larger cloud-init files; arm64/Graviton support and Ubuntu 26.04 LTS in the curated AMI/instance-type lists; single-resource detail views for an individual EC2 instance and AMI (Show instance detail/Show AMI detail), alongside the existing per-resource-type list views; and a sixth domain, Configuration, to view and edit clasm's own ~/.clasm settings (regions, backup directory rules, Origin tag config) from within clasm instead of hand-editing YAML, with changes staged in memory until an explicit Save. Finally, every domain menu's item ordering was reviewed and regrouped (View/Inspect -> Instance -> AMI -> Launch Template -> Maintenance for Compute, matching the List -> Detail -> Create -> CRUD shape IAM already used), and several menu labels were unified onto consistent terminology ('List S3 Buckets' -> 'Show Buckets', 'Show resource lists' -> 'Show Key Pairs', 'View Role/Instance Profile Detail' -> 'Show Role/Instance Profile Detail').

Full Changelog: v0.0.4...v0.0.5

v0.0.4

Adds configurable EBS root volume size: set the size when creating an instance or launch template (instead of always inheriting the AMI's default), and a new 'Resize instance's root volume' menu entry that grows a running instance's EBS volume and automates the OS-level partition/filesystem growth via SSM, falling back to printed manual instructions whenever that automation can't proceed safely. Also a bug fix, found via live testing of the above: any status or error text printed immediately before a domain menu's next full-height selection screen was getting wiped from view before it could be read -- affected every dispatched action's own output (success or failure) and every refresh error, across all four domains (Compute, S3, Key Management, Tag Management). Every menu loop now pauses for an explicit Enter before redrawing, so operators can actually read what an action reported.

Full Changelog: v0.0.3...v0.0.4

v0.0.3

Bug fix release. Cancelling ('q') the S3 bucket picker inside Backup Archive & Trim exited the whole program instead of returning to the previous menu -- a regression, since cancelling the instance picker one step earlier in the same workflow already backed out correctly. backupArchiveAndTrim's bucket-selection step now maps that cancellation the same way, with a regression test covering it. Also improves every Menu-tier huh.Select picker's on-screen hint (bucket/AMI/key-pair/launch-template selection, and every domain/action menu) to mention '/' to filter alongside 'q' to cancel/go back/exit, so it's clear before typing that a bare 'q' quits rather than filters; the hint wording is now centralized as shared constants so it can't drift out of sync again. No functional changes beyond these two fixes -- all v0.0.2 functionality is unchanged.

Full Changelog: v0.0.2...v0.0.3