This repository was archived by the owner on Oct 8, 2025. It is now read-only.
Merged
Conversation
* Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs.
* Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work.
…159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role.
* Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp.
* Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var.
* Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once.
* Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role.
* Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild.
Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com>
* Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too.
* cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults
* Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf7.
* Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs.
* cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var
* previous_cachetool_remove_before_install_new * namespace fix
* fix_cachetool_removal_step * fix task name
* fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task
* Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change.
* Fixing sync jobs. * Fixing bad task title.
* Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir.
* Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'.
* change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo
* cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2
* Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling.
…fixes-PR-devel-1.x
…fixes-PR-devel-1.x
* Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Fixing docs and CI. * Updating the main README in docs. * Adding cron user back in, but omitted by default and commented out. * Adding --set-gtid-purged=OFF param to default mysqldump command. * Revert "Adding --set-gtid-purged=OFF param to default mysqldump command." This reverts commit 972d8ff. * Fixing linting error (no upper case chars in var names).
* Merging 1.x down into devel (#269) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Target host checking pr devel (#270) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Sync enhancements pr devel (#272) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Bug fixes pr devel (#274) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Revert "Bug fixes pr devel (#274)" (#276) This reverts commit 3e2977500ee9fb68c8e5b01e836e07718549ab66. * Bug fixes pr devel (#277) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make conf…
* Moving LHCI test role to tests subdirectory. * Updating documentation. * Full namespacing added. * Updating composer command role to support arguments. * Adding PHPUnit testing role. * Making installation of phpunit with composer an option. * Suppressing failure if phpunit.xml file isn't found.
* Moving LHCI test role to tests subdirectory. * Updating documentation. * Full namespacing added. * Updating composer command role to support arguments. * Adding PHPUnit testing role. * Making installation of phpunit with composer an option. * Suppressing failure if phpunit.xml file isn't found. * Switching to deploy_path for checking built code. * Moving composer command code to a block. * Fixing the phpunit.xml.dist lookup. * We need ignore_errors in this case. * Somehow my 'remote_src: true' line got lost in conflicts! * Fixing default paths to use Drupal default phpunit config by default. * Using 'skip' argument for first_found. * Updating phpunit docs. * Updating phpunit docs. * Adding a diffy role under tests. * Forgot to update variable name. * Fixing docs typo.
* First pass at waiting for healthy only instances. * Moving the delegate_to and run_once to the block. * Adding AnsibleUnicode as a possible var type. * Fixing or logic in when clause for legacy strings. * Reverting to using command and CLI for ASG processes, simpler for IAM perms. * Fixing region variable for AWS. * Fine tuning when ASG manipulation occurs. * Moving ASG parts to another role so we can call them separately. * Adding a pause to not hammer the API. * Missed a variable update. * Adding some debug lines. * Fixing boto profile handling. * Adding extra build_completed flag to control when ASG processes are altered. * Docs and switching to using a lock file instead of a variable, which doesn't traverse different plays. * Suppressing cachetool installation on deploy servers in example play. * Docs slight improvement. * Slightly improved ASG handling, dynamic by build_type. * Re-adding reminder to refactor. * Adding verbose option for ASG role. * Simplifying role by removing dependency on lock files. * Verbose var now obsolete.
* Moving LHCI test role to tests subdirectory. * Updating documentation. * Full namespacing added. * Updating composer command role to support arguments. * Adding PHPUnit testing role. * Making installation of phpunit with composer an option. * Suppressing failure if phpunit.xml file isn't found. * Switching to deploy_path for checking built code. * Moving composer command code to a block. * Fixing the phpunit.xml.dist lookup. * We need ignore_errors in this case. * Somehow my 'remote_src: true' line got lost in conflicts! * Fixing default paths to use Drupal default phpunit config by default. * Using 'skip' argument for first_found. * Updating phpunit docs. * Updating phpunit docs. * Adding a diffy role under tests. * Forgot to update variable name. * Fixing docs typo. * Adding a PHP CodeSniffer testing role. * Documentation enhancements.
* Moving LHCI test role to tests subdirectory. * Updating documentation. * Full namespacing added. * Updating composer command role to support arguments. * Adding PHPUnit testing role. * Making installation of phpunit with composer an option. * Suppressing failure if phpunit.xml file isn't found. * Switching to deploy_path for checking built code. * Moving composer command code to a block. * Fixing the phpunit.xml.dist lookup. * We need ignore_errors in this case. * Somehow my 'remote_src: true' line got lost in conflicts! * Fixing default paths to use Drupal default phpunit config by default. * Using 'skip' argument for first_found. * Updating phpunit docs. * Updating phpunit docs. * Adding a diffy role under tests. * Forgot to update variable name. * Fixing docs typo. * Adding a PHP CodeSniffer testing role. * Documentation enhancements. * Fixing bug where stdout of diffy is overwritten.
* Refactoring the deploy_container role. * scheduled_containers * delegate tasks where aws profile is needed * fix list-string for some vars and rename role --------- Co-authored-by: Greg Harvey <greg.harvey@gmail.com>
* Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster. * Fixing docs and CI. * Updating the main README in docs. * Adding cron user back in, but omitted by default and commented out. * Adding --set-gtid-purged=OFF param to default mysqldump command. * Revert "Adding --set-gtid-purged=OFF param to default mysqldump command." This reverts commit 972d8ff. * Fixing linting error (no upper case chars in var names). * Stopping builds if the track file and build path do not match. * Trying casting build IDs as integers for comparison.
* scheduled_containers_fix * scheduled_containers_fix * fix namespaces
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.