Skip to content

Prefix dump scripts with keyed resource name#12

Merged
stevethomas merged 24 commits intomainfrom
steve/prefix-dump-scripts
Mar 26, 2026
Merged

Prefix dump scripts with keyed resource name#12
stevethomas merged 24 commits intomainfrom
steve/prefix-dump-scripts

Conversation

@stevethomas
Copy link
Copy Markdown
Member

@stevethomas stevethomas commented Mar 26, 2026

Summary

  • Namespace dump scripts and output per app to prevent collisions when multiple YOLO apps share a scheduler instance
  • Scripts and dumps live under ~/yolo/{keyed-resource-name}/ (e.g. ~/yolo/yolo-production-lp-app/mysqlbackup.sh, ~/yolo/yolo-production-lp-app/dumps/)
  • Separated mysqlbackup cron from the scheduler stub — backup step now owns its own /etc/cron.d/ entry via a dedicated cron stub
  • Backup output logged to /var/log/yolo/{keyed-resource-name}/mysqlbackup.log instead of /dev/null, with logrotate (weekly, 12 rotations)
  • Scripts always written regardless of mysqldump flag — only the cron entry is conditional
  • New ProvisionDirectoriesStep runs first in StartCommand to create ~/yolo/{app}/ and /var/log/yolo/{app}/
  • SetOwnershipAndPermissionsStep moved to run last (before restart) — chowns /home/ubuntu, /var/log/yolo, and /var/www
  • Paths::yoloDir() and Paths::logDir() helpers centralise path construction

Anything the reviewer needs to know

  • The old mysqlbackup cron line in the scheduler stub gets overwritten on deploy — no action needed
  • These orphaned files should be cleaned up manually on scheduler instances after deploy:
    rm -f /home/ubuntu/mysqlbackup.sh /home/ubuntu/mysqldump-table.sh
    rm -rf /home/ubuntu/dumps
    

🤖 Generated with Claude Code

stevethomas and others added 4 commits March 26, 2026 11:29
Prevents multiple YOLO apps on the same scheduler instance from
overwriting each other's dump scripts and temp files. Scripts and
dumps now live under /home/ubuntu/yolo/ with app-specific filenames
via keyedResourceName (e.g. yolo-production-lp-app-mysqlbackup.sh).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The dump cron is a dump concern, not a scheduler concern. The backup
step now owns its own /etc/cron.d/ entry with a keyed resource name,
removing the MYSQLBACKUP_PATH placeholder from the scheduler stub.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use ~/yolo-{env}-{app}/ as the app directory so scripts get simple
names and dumps are isolated per app too.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/Steps/Start/Scheduler/SyncMysqlBackupStep.php Outdated
stevethomas and others added 6 commits March 26, 2026 12:12
The directory is created by root during deploy but the cron runs as
ubuntu — without chown, ubuntu can't create the dumps subdirectory.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chown -R /home/ubuntu in the dedicated step instead of individual
chowns scattered across dump and bash profile steps.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Follow the Nightwatch pattern — unlink files on skip so disabling
the feature in the manifest actually removes the artefacts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use file_exists/is_dir checks consistent with Nightwatch pattern.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Keeps home directory clean — all YOLO artefacts live under
/home/ubuntu/yolo/{keyed-resource-name}/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/Steps/Start/All/SetOwnershipAndPermissionsStep.php
stevethomas

This comment was marked as outdated.

stevethomas and others added 2 commits March 26, 2026 14:50
The chown -R /home/ubuntu was running before the dump steps created
root-owned directories. Moving it to just before RestartServicesStep
ensures all files written during the start phase get chowned.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cron now logs to /var/log/yolo/{keyed-resource-name}/mysqlbackup.log
instead of /dev/null. Added logrotate config for yolo logs (weekly,
12 rotations). Backup step creates the log directory on sync.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/Steps/Start/Scheduler/SyncMysqlBackupStep.php
stevethomas and others added 2 commits March 26, 2026 15:28
One shared /etc/logrotate.d/yolo config with /var/log/yolo/*/*.log
covers all apps — no per-app logrotate entry needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Avoids confusion with the app directory at /var/www/{name}.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/Steps/Start/Scheduler/SyncMysqlBackupStep.php Outdated
stevethomas and others added 10 commits March 26, 2026 15:33
Centralise path construction instead of inline sprintf in each step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Log directory is created by root but cron writes as ubuntu. Guarded
with test -d since the directory only exists when mysqldump is enabled.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Scripts are harmless without the cron — always write them so they're
available for manual use. Only the cron entry is conditional. Removes
cleanup logic and simplifies both steps. Table dump step has no
conditional at all now.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mkdir in SetOwnershipAndPermissionsStep so the log directory exists
on all server groups, not just scheduler. Removes the test -d guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Create ~/yolo/{app}/ and /var/log/yolo/{app}/ once in the ownership
step so dump steps just write files like everything else.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Runs first in StartCommand to create ~/yolo/{app}/ and
/var/log/yolo/{app}/ before any file-writing steps.
SetOwnershipAndPermissionsStep stays last for chown.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Both dump scripts should be executable for direct invocation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Scripts are always invoked via bash, chmod unnecessary. Added
StartCommand step ordering and filesystem conventions to CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Don't chown the entire /var/www — just the app's directory.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment thread src/Steps/Start/Scheduler/SyncMysqlBackupStep.php
Copy link
Copy Markdown
Member Author

@stevethomas stevethomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Verdict: ✅ Approved

Risk Assessment

Category Status
Data Integrity
Security
Breaking Changes
Operational ⚠️
Test Coverage

Findings

⚠️ Warnings

Rollback scenario: deploying this creates a keyed cron file at /etc/cron.d/{app}-mysqlbackup. Rolling back to old code rewrites the scheduler stub with the embedded backup line but does not remove the keyed cron file — both fire at 09:00, producing duplicate S3 uploads. Add a rollback note to the PR description.


🤖 Reviewed by Claude Code

@stevethomas stevethomas merged commit 238640c into main Mar 26, 2026
3 checks passed
@stevethomas stevethomas deleted the steve/prefix-dump-scripts branch March 26, 2026 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant