Drush refactor pr devel (#235)

* Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94)

* Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs).

* Set up nightly backup crons in separate files in /etc/cron.d

* Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms.

* Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump.

* Use Ansible modules to look up RDS host and to copy the nightly backup into place.

* Delegate PATH setup in db backup cron to localhost.

* Shell bad. Command good. But makes it convoluted. Oh well.

* Used wrong database name in source database copy.

* Try and disable the ReplaceUnhealthy auto scale process during syncs.

* Fix Drupal cron roles when deferring to deploy server.

* Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97)

* Adding a new SimpleSAMLphp meta role. (#100)

* Allowing users to set cachetool version properly. (#102)

* Deploy ami pr 1.x (#106)

* Including an ami.yml for packing AMIs on build finale.

* New api_call role, focused on GitLab for now.

* Optionally trigger an infra build with an API call.

* Making the MySQL dump command for routine back-ups less aggressive. (#104)

* Making the MySQL dump command for routine back-ups less aggressive.

* Making max_allowed_packets a variable we can set.

* Adding dump command to a re-usable central var.

* Renaming _init var so it's easier to distinguish in the code.

* Fix database backups pr 1.x (#109)

* Making the MySQL dump command for routine back-ups less aggressive.

* Making max_allowed_packets a variable we can set.

* Adding dump command to a re-usable central var.

* Renaming _init var so it's easier to distinguish in the code.

* Defaults file must be the first param for mysqldump.

* Fix MySQL backup deferral. (#110)

* Files recurse fix pr 1.x (#112)

* Don't recurse through site directory when setting permissions during config_generate step.

* Update drupal7 config_generate perms update task and use true/false instead of yes/no.

* Improve multisite support (#115)

* Pass -l option in drush commands to specify site name, which should match the site folder name.

* Move -l option in drush commands to the first option instead of the last.

* Static credentials handling fix pr 1.x (#119)

* Attempt to fix static credentials handling when deploying to an ASG.

* Include build_type in static password file location.

* Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now.

* Add a couple of comments to legacy handling of static credentials.

* Making contents of deploy tar 'ownerless'. (#117)

* Implement file syncing (#124)

* Add a files_sync role.

* Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role.

* Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself.

* Use rsync instead of copy when syncing files to target server as copy is so slow.

* Ensure file sync tasks are run only once.

* Create Drupal-specific sync roles (#128)

* Add a drupal_sync_tasks role to run Drupal specific tasks during syncs.

* Move sync database_apply files into tasks subdirectory.

* Woops, used bad role names in sync database update roles.

* Remove feature reverting from drupal7 database_apply sync role.

* Fixing GRANT query for MySQL > 8.0. (#131)

* Fixing GRANT query for MySQL > 8.0.

* Create what though?

* Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133)

* Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149)

* Squashfs pr 1.x (#150)

* Adding SquashFS option to syncing.

* Make 'tarball' the default mount type so nothing existing breaks.

* Altering deploy_path so we can build in another location for SquashFS builds.

* Ensuring the build dir exists if doing a SquashFS build.

* Force symlink creation as the deploy dir may not exist yet.

* Running mount commands with sudo.

* Tweaking link command and destination for live links.

* Slight bug in link path handling for SquashFS.

* Checking for existing mount and using remount operation.

* Stop ce-deploy trying to delete from read-only SquashFS mount.

* Formatting error, these special jinja2 things are not filters - no spaces.

* Reloading services to make sure mounting doesn't fail.

* Picking more sensible loop var name.

* Moving to shell for mount check and fixing jinja2 filter names.

* Trying with the posix mount module instead of command.

* Working through user/sudo issues.

* Altering symlink handling slightly so we have the deploy directory always set.

* Simplifying 'when' checks.

* Check deploy_code.mount_type is defined when setting facts in init role. (#155)

* Make config imports during syncs optional (#157)

* Use a different variable for config imports during a sync so they can be optional.

* Actually, no. Use a variable to determine if config should be imported during a sync.

* Fix some logic with config import during syncs and add a comment to explain when the task runs.

* Squashfs pr 1.x (#153)

* Adding SquashFS option to syncing.

* Make 'tarball' the default mount type so nothing existing breaks.

* Altering deploy_path so we can build in another location for SquashFS builds.

* Ensuring the build dir exists if doing a SquashFS build.

* Force symlink creation as the deploy dir may not exist yet.

* Running mount commands with sudo.

* Tweaking link command and destination for live links.

* Slight bug in link path handling for SquashFS.

* Checking for existing mount and using remount operation.

* Stop ce-deploy trying to delete from read-only SquashFS mount.

* Formatting error, these special jinja2 things are not filters - no spaces.

* Reloading services to make sure mounting doesn't fail.

* Picking more sensible loop var name.

* Moving to shell for mount check and fixing jinja2 filter names.

* Trying with the posix mount module instead of command.

* Working through user/sudo issues.

* Altering symlink handling slightly so we have the deploy directory always set.

* Simplifying 'when' checks.

* Adding a revert behaviour for SquashFS.

* Making some wording a little less ambiguous.

* Fixing bad image path and adding clauses to stat check.

* Trying to make remounting SquashFS images a bit safer.

* Adding when clauses to SquashFS image path check.

* Removing revert code, as it cannot work.

* Add cache clears to Drupal deployments, before DB updates and stuff (#159)

* Add cache clears to Drupal deployments, before database updates and config import are run.

* Move cache clears out of meta roles and only run in database_apply role on non-initial builds.

* Add cache clear stuff to drupal7 database_apply role.

* Avoid leaving exponentially growing sqsh files in build locations! (#164)

* Avoid leaving exponentially growing sqsh files in build locations!

* Use mount point instead of /tmp for stowing previous sqsh file.

* Exclude sqsh file pr 1.x (#167)

* Avoid leaving exponentially growing sqsh files in build locations!

* Use mount point instead of /tmp for stowing previous sqsh file.

* Making sure image/tarball filenames are build specific in /tmp.

* Removing unnecessary lines in Drupal config generation. (#169)

* Removing unnecessary lines in Drupal config generation.

* Drupal 7 lacked the install_command var.

* Ensuring dump directory exists on backup step. (#172)

* Ensuring dump directory exists on backup step.

* Acts on shared storage, so can run once.

* Allowing Drupal 7 jobs to disable cron. (#174)

* Suppress db revert pr 1.x (#177)

* Supporting option to suppress reverting backups.

* Fixing namespacing in database_backup role.

* Fixing bad assumption that databases will have TCP connections. (#179)

* Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180)

* Handling the 'drush deploy' command more elegantly for Drupal 8+.

* Changing logic to AND for cache rebuild.

* Attempt to clear the opcache during Drupal deployments. (#182)

Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com>

* Better drush deploy support pr 1.x (#185)

* Handling the 'drush deploy' command more elegantly for Drupal 8+.

* Changing logic to AND for cache rebuild.

* Em's clear cache code needs to run for each 'site' too.

* Cron job schedule params pr 1.x (#190)

* cron job schedule params, namespaces

* drupal7-cron-params

* cron-job-schedule-params-fix-defaults

* Adding option to stop services that might interfere with a squashfs mount. (#193)

* Drush refactor pr 1.x (#197)

* Updating docs.

* Consistent drush handling in cron.

* Linting cruft and removing unimplemented drush.yml handling from D7.

* Refactor drush role to support new GitHub release format.

* Updating drush README.

* Casting the first part of the drush version string as an integer.

* Refactoring integer handling.

* Major version string will always be a string, assuming below 8 will never exist.

* Downloaded drush archive is remote, not local.

* Refactoring handling of .bin directory.

* Supporting drush-launcher.

* Removing drush installation support for Drupal 8 and above.

* Bad variable name in Drupal cron jobs.

* Fixing maintenance mode command linting and consistency.

* Moving chdir to args to see if it helps.

* Trying an actual 'cd' command inline.

* Cannot '&&' in a cmd.

* Trying drush's --root flag to set Drupal path.

* With --root we do not need chdir for drush.

* Revert "With --root we do not need chdir for drush."

This reverts commit d07aaf7.

* Better deploy_code role docs. (#213)

* Better deploy_code role docs.

* roles path error in docs.

* roles path error in docs.

* Adding a note about deploy_previous handling for squashfs.

* Cachetool settings and drupal7 tasks fix pr 1.x (#211)

* cachetool_settings_and_drupal7_tasks_fix

* cachetool_settings_and_drupal7_tasks_fix

* cachetool_settings_and_drupal7_tasks_fix

* cachetool settings fixing condition and namespace

* fixing cachetool adapter options

* fixed var

* fixed var

* fixed var

* previous_cachetool_remove_before_install_new (#224)

* previous_cachetool_remove_before_install_new

* namespace fix

* Fix cachetool removal pr 1.x (#227)

* fix_cachetool_removal_step

* fix task name

* Fix cachetool removal pr 1.x (#229)

* fix_cachetool_removal_step

* fix task name

* fix_cachetool_removal_task

* fix_cachetool_removal_task

* Fixing sync jobs.

* Documentation enhancements pr 1.x (#218)

* Better deploy_code role docs.

* roles path error in docs.

* roles path error in docs.

* Adding a note about deploy_previous handling for squashfs.

* Reference incorrect role for deploy user sudo perms.

* Minor edits to frontpage README.

* Rebuilt docs.

* Accidentally overwrote docs change.

* Fixing bad task title.

* Drush refactor pr 1.x (#231)

* Fixing sync jobs.

* Fixing bad task title.

* Trying to fix D7 drush issues with chdir.

* Moving D7 drush commands back to 'shell'.

---------

Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com>
Co-authored-by: Dionisio <dionisiofernandez83@gmail.com>
Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com>

Author: 4 people

roles/database_apply/database_apply-drupal7/tasks/ctools.yml

@@ -1,12 +1,13 @@
 ---
+# For some reason D7 drush doesn't respect 'chdir' with command, using shell instead.
 - name: Check if Ctools module is enabled.
   ansible.builtin.shell:
     cmd: "{{ drush_bin }} pm-info ctools | grep ':  enabled' | wc -l"
     chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
   register: ctools_enabled
 
 - name: Revert Drupal configuration from Ctools.
-  ansible.builtin.command:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -y ctools-export-revert --all"
     chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
   when: ctools_enabled.stdout == "1"

roles/database_apply/database_apply-drupal7/tasks/features.yml

@@ -1,8 +1,9 @@
 ---
+# For some reason D7 drush doesn't respect 'chdir' with command, using shell instead.
 - name: Check if Features module is enabled.
-  shell: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drush_bin }} -l {{ site.folder }} pm-info features | grep ':  enabled' | wc -l"
+  ansible.builtin.shell: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drush_bin }} -l {{ site.folder }} pm-info features | grep ':  enabled' | wc -l"
   register: features_enabled
 
 - name: Revert Drupal configuration from Features.
-  shell: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drush_bin }} -l {{ site.folder }} -y {{ site.revert_features_command }}"
+  ansible.builtin.shell: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drush_bin }} -l {{ site.folder }} -y {{ site.revert_features_command }}"
   when: features_enabled.stdout == "1"

roles/database_apply/database_apply-drupal7/tasks/main.yml

@@ -1,7 +1,7 @@
 ---
-
+# For some reason D7 drush doesn't respect 'chdir' with command, using shell instead.
 - name: Install Drupal.
-  shell:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -l {{ site.folder }} {{ site.install_command }}"
     chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
   become: "{{ 'no' if www_user == deploy_user else 'yes' }}"
@@ -12,7 +12,7 @@
   when: previous_build_number == 0 or (site.force_install is defined and site.force_install)
 
 - name: Fix permissions on Drupal directory.
-  shell:
+  ansible.builtin.shell:
     cmd: "chmod 755 {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
   with_items: "{{ drupal.sites }}"
   loop_control:
@@ -30,15 +30,15 @@
   when: previous_build_number > 0
 
 - name: Apply Drupal database updates.
-  shell:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -l {{ site.folder }} -y updb"
     chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
   with_items: "{{ drupal.sites }}"
   loop_control:
     loop_var: site
 
 - name: Revert Drupal configuration with Features.
-  include_tasks: features.yml
+  ansible.builtin.include_tasks: features.yml
   with_items: "{{ drupal.sites }}"
   loop_control:
     loop_var: site
@@ -47,7 +47,7 @@
     - site.revert_features_command
 
 - name: Revert Drupal configuration with Ctools.
-  include_tasks: ctools.yml
+  ansible.builtin.include_tasks: ctools.yml
   with_items: "{{ drupal.sites }}"
   loop_control:
     loop_var: site

roles/sanitize/admin_creds/admin_creds-drupal7/tasks/admin.yml

@@ -4,11 +4,12 @@
 - ansible.builtin.set_fact:
     _admin_pwd: "{{ lookup('password', '/tmp/{{ project_name }}-{{ site.folder }}-{{ build_type }}-{{ build_number }}-pwd chars=ascii_letters') }}"
 
+# For some reason D7 drush doesn't respect 'chdir' with command, using shell instead.
 - name: Reset admin username.
-  ansible.builtin.command:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -l {{ site.folder }} sql-query \"UPDATE users SET name='{{ _admin_user }}' WHERE uid=1;\""
     chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
 - name: Reset admin password.
-  ansible.builtin.command:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -l {{ site.folder }} upwd {{ _admin_user }} --password='{{ _admin_pwd }}'"
     chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"

roles/sync/drupal_sync_tasks/cache_clear/cache_clear-drupal7/tasks/main.yml

@@ -1,6 +1,7 @@
 ---
+# For some reason D7 drush doesn't respect 'chdir' with command, using shell instead.
 - name: Clear Drupal 7 cache.
-  ansible.builtin.command:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -l {{ site.folder }} -y cc all"
     chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}"
   become: "{{ 'no' if www_user == deploy_user else 'yes' }}"

roles/sync/drupal_sync_tasks/database_apply/database_apply-drupal7/tasks/main.yml

@@ -1,6 +1,7 @@
 ---
+# For some reason D7 drush doesn't respect 'chdir' with command, using shell instead.
 - name: Apply Drupal database updates.
-  ansible.builtin.command:
+  ansible.builtin.shell:
     cmd: "{{ drush_bin }} -l {{ site.folder }} -y updb"
     chdir: "{{ live_symlink_dest }}/{{ webroot }}/sites/{{ site.folder }}"
   with_items: "{{ drupal.sites }}"