Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
3ccfe51
Defer nightly backups, disable ASG processes during syncs and run syn…
EmlynK Jan 20, 2022
a54c09c
Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97)
DionisioFG Mar 10, 2022
223a3b5
Adding a new SimpleSAMLphp meta role. (#100)
gregharvey Mar 31, 2022
32843d5
Allowing users to set cachetool version properly. (#102)
gregharvey Apr 13, 2022
91ea64e
Deploy ami pr 1.x (#106)
gregharvey Apr 14, 2022
371b6b9
Making the MySQL dump command for routine back-ups less aggressive. (…
gregharvey Apr 20, 2022
a9cc7ad
Fix database backups pr 1.x (#109)
gregharvey Apr 20, 2022
4c4c3ad
Fix MySQL backup deferral. (#110)
EmlynK Apr 22, 2022
029848d
Files recurse fix pr 1.x (#112)
EmlynK Apr 26, 2022
b71fa83
Improve multisite support (#115)
EmlynK Apr 29, 2022
c933dc8
Static credentials handling fix pr 1.x (#119)
EmlynK May 23, 2022
b6ea020
Making contents of deploy tar 'ownerless'. (#117)
gregharvey May 31, 2022
1c2b206
Implement file syncing (#124)
EmlynK Jun 7, 2022
1cd0ed0
Create Drupal-specific sync roles (#128)
EmlynK Jun 8, 2022
904033f
Fixing GRANT query for MySQL > 8.0. (#131)
gregharvey Jun 10, 2022
94715d8
Use IF NOT EXISTS when creating database user as that command fails i…
EmlynK Jun 13, 2022
42f12c8
Attempt to fix syncs whenever the 'dump' type is used for source or t…
EmlynK Jun 16, 2022
a9c6abf
Squashfs pr 1.x (#150)
gregharvey Jun 20, 2022
a4de771
Check deploy_code.mount_type is defined when setting facts in init ro…
EmlynK Jun 24, 2022
fe2aaf1
Make config imports during syncs optional (#157)
EmlynK Jun 28, 2022
451227d
Squashfs pr 1.x (#153)
gregharvey Jun 29, 2022
c7aae71
Add cache clears to Drupal deployments, before DB updates and stuff (…
EmlynK Aug 16, 2022
15b8daa
Avoid leaving exponentially growing sqsh files in build locations! (#…
gregharvey Sep 5, 2022
5cab4ae
Exclude sqsh file pr 1.x (#167)
gregharvey Oct 6, 2022
459faba
Removing unnecessary lines in Drupal config generation. (#169)
gregharvey Oct 7, 2022
200f590
Ensuring dump directory exists on backup step. (#172)
gregharvey Oct 10, 2022
2c74432
Allowing Drupal 7 jobs to disable cron. (#174)
gregharvey Oct 14, 2022
0385211
Suppress db revert pr 1.x (#177)
gregharvey Oct 14, 2022
86f138a
Fixing bad assumption that databases will have TCP connections. (#179)
gregharvey Nov 18, 2022
b236540
Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180)
gregharvey Dec 9, 2022
58069b6
Attempt to clear the opcache during Drupal deployments. (#182)
gregharvey Dec 9, 2022
3ea0bfe
Better drush deploy support pr 1.x (#185)
gregharvey Dec 9, 2022
b835344
Cron job schedule params pr 1.x (#190)
tymofiisobchenko Dec 30, 2022
e97e852
Adding option to stop services that might interfere with a squashfs m…
gregharvey Jan 23, 2023
7bbb77f
Drush refactor pr 1.x (#197)
gregharvey Jan 25, 2023
b0f4033
Better deploy_code role docs.
gregharvey Jan 27, 2023
b67401b
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Jan 27, 2023
d5c9a17
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
d0a0ed5
roles path error in docs.
gregharvey Jan 27, 2023
f31d4fd
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Jan 27, 2023
705f73b
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
f71cd25
roles path error in docs.
gregharvey Jan 27, 2023
334c43b
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Jan 27, 2023
e0e374f
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
9a56fa2
Adding a note about deploy_previous handling for squashfs.
gregharvey Jan 27, 2023
eeed6da
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Jan 27, 2023
5cf6847
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
cc31f6a
Better deploy_code role docs. (#213)
gregharvey Jan 27, 2023
a0a8a3e
Reference incorrect role for deploy user sudo perms.
gregharvey Jan 27, 2023
c8f95d6
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Jan 27, 2023
d429ff9
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
f6a3cb3
Cachetool settings and drupal7 tasks fix pr 1.x (#211)
tymofiisobchenko Jan 27, 2023
ac6d02a
Minor edits to frontpage README.
gregharvey Jan 27, 2023
3320333
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Jan 27, 2023
4819eb5
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
a2cfcef
Rebuilt docs.
gregharvey Jan 27, 2023
36f79ca
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Jan 27, 2023
44fef49
previous_cachetool_remove_before_install_new (#224)
tymofiisobchenko Jan 31, 2023
f1b2b90
Fix cachetool removal pr 1.x (#227)
tymofiisobchenko Jan 31, 2023
de6d80b
Fix cachetool removal pr 1.x (#229)
tymofiisobchenko Feb 1, 2023
cbe8841
Documentation and linting fixes.
gregharvey Feb 3, 2023
78e9cf4
Merge branch 'devel' into documentation_enhancements-PR-devel
gregharvey Feb 3, 2023
4ffa9a8
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Feb 3, 2023
cc6e584
Accidentally overwrote docs change.
gregharvey Feb 3, 2023
33ffdd3
Merge branch 'documentation_enhancements' into documentation_enhancem…
gregharvey Feb 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
# ce-deploy

[![Build Status](https://api.travis-ci.com/codeenigma/ce-deploy.svg?branch=1.x)](https://api.travis-ci.com/codeenigma/ce-deploy.svg?branch=1.x)

A set of Ansible roles and wrapper scripts to deploy (web) applications.

## Overview
The "stack" from this repo is to be installed on a "deploy" server/runner, to be used in conjonction with a CI/CD tool (Jenkins, Gitlab, Travis, ...).
It allows the deploy steps for a given app to be easily customizable at will, and to be stored alongside the codebase of the project.
It allows the deploy steps for a given app to be easily customisable and to be stored alongside the codebase of the project.
When triggered from a deployment tool, the stack will clone the codebase and "play" a given deploy playbook from there.

<!--TOC-->
Expand Down
2 changes: 1 addition & 1 deletion docs/roles/cache_clear/cache_clear-opcache.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ cache_clear_opcache:
# eg.
# --fcgi=127.0.0.1:9000
# Leave blank to use /etc/cachetool.yml
adapter: ""
# adapter: "127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version.
# Bins to clear.
clear_opcache: true
clear_apcu: false
Expand Down
3 changes: 2 additions & 1 deletion docs/roles/cli/cachetool.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Installs the `drush` command-line tool for the deploy user.
```yaml
---
cachetool:
version: latest # # enter three-digit version number, e.g. "7.0.0", to install a specific version
version: "" # enter three-digit version number, e.g. "7.0.0", to install a specific version. If not specified, will be installed depending on the php version.

```

<!--ENDROLEVARS-->
6 changes: 3 additions & 3 deletions docs/roles/deploy_code.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ With this method the live code directory is also the build directory, therefore
## `squashfs` builds
Because `tarball` is very slow, we have a second method using [`squashfs`](https://github.com/plougher/squashfs-tools). This filesystem is designed for packing and compressing files into read-only images - initially to deploy to removable media - that can simply be mounted, similar to a macOS Apple Disk Image (DWG) file. It is both faster to pack than a tarball *and* instant to deploy (it's just a `mount` command).

However, the build process is more complex. Because mounted `squashfs` images are read only, we cannot build over them as we do in other types of build. [We alter the build path variables in the `_init` role](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/_init/tasks/main.yml#L25) so the build happens in a separate place and then in the `cleanup.yml` we pack the built code into an image ready to be deployed. Again, because the images are read-only mounts, the live site needs to be *unmounted* with an `umount` command and then remounted with a `mount` command to be completely deployed. This requires the `ce-deploy` user to have extra `sudo` permissions, which is handled by [the `squashfs` role in `ce-provision`](https://github.com/codeenigma/ce-provision/blob/1.x/roles/squashfs)
However, the build process is more complex. Because mounted `squashfs` images are read only, we cannot build over them as we do in other types of build. [We alter the build path variables in the `_init` role](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/_init/tasks/main.yml#L25) so the build happens in a separate place and then in the `cleanup.yml` we pack the built code into an image ready to be deployed. Again, because the images are read-only mounts, the live site needs to be *unmounted* with an `umount` command and then remounted with a `mount` command to be completely deployed. This requires the `ce-deploy` user to have extra `sudo` permissions, which is handled by [the `squashfs` role in `ce-provision`](https://github.com/codeenigma/ce-provision/tree/1.x/roles/squashfs).

Consequently, at the build stage there are two important extra variables to set:

Expand All @@ -63,10 +63,10 @@ deploy_code:

`service_action` is whether `ce-deploy` should restart the services in the list of stop them, unmount and remount the image and start them again. The latter is the only "safe" way to deploy, but results in a second or two of down time.

Finally, as with the `tarball` method, the packed image is copied up to the NAS to be available to all future servers and is always named `deploy.sqsh`. The previous codebase is *also* packed and copied to the NAS, named `deploy_previous.sqsh` in the same directory.
Finally, as with the `tarball` method, the packed image is copied up to the NAS to be available to all future servers and is named `PROJECTNAME_BUILDTYPE_deploy.sqsh`. [See the code here.](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/deploy_code/tasks/cleanup.yml#L80) The previous codebase is *also* packed and copied to the NAS, named `PROJECTNAME_BUILDTYPE_deploy_previous.sqsh` in the same directory.

### Rolling back
Rolling back from a bad `squashfs` build means copying `deploy_previous.sqsh` down from the NAS to a sensible location in the `ce-deploy` user's home directory, unmounting the current image and mounting `deploy_previous.sqsh` in its place. Once you've done that, to ensure future autoscaling events do not load the bad code, on the NAS you will need to rename `deploy.sqsh` to something else (or delete it entirely if you're sure you don't want it) and rename `deploy_previous.sqsh` as `deploy.sqsh`, so it is used on an autoscale event.
Rolling back from a bad `squashfs` build means copying the project/branch specific `deploy_previous.sqsh` down from the NAS to a sensible location in the `ce-deploy` user's home directory, unmounting the current image and mounting the `deploy_previous.sqsh` in its place. Once you've done that, to ensure future autoscaling events do not load the bad code, on the NAS you will need to rename `deploy.sqsh` to something else (or delete it entirely if you're sure you don't want it) and rename your `deploy_previous.sqsh` as `PROJECTNAME_BUILDTYPE_deploy.sqsh`, so it is used on an autoscale event. [You can see the actual code that runs on autoscale events here.](https://github.com/codeenigma/ce-provision/blob/1.x/roles/mount_sync/templates/init-squashfs.sh.j2)

Same as with the `tarball` method, as long as the `database_backup` is using the `rolling` method then the "roll back" database will still exist and the credentials will be correct in the `deploy_previous.sqsh` image. Again, if the backup method is `dump` then you will need to inspect [the `mysql_backup.dumps_directory` variable](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/database_backup/database_backup-mysql/defaults/main.yml#L4) to see where the backup was saved in order to restore it.

Expand Down
2 changes: 1 addition & 1 deletion roles/cache_clear/cache_clear-opcache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ cache_clear_opcache:
# eg.
# --fcgi=127.0.0.1:9000
# Leave blank to use /etc/cachetool.yml
adapter: ""
# adapter: "127.0.0.1:9081" # Leave commented to automatically detect the adapter based on PHP version.
# Bins to clear.
clear_opcache: true
clear_apcu: false
Expand Down
3 changes: 2 additions & 1 deletion roles/cli/cachetool/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ Installs the `drush` command-line tool for the deploy user.
```yaml
---
cachetool:
version: latest # # enter three-digit version number, e.g. "7.0.0", to install a specific version
version: "" # enter three-digit version number, e.g. "7.0.0", to install a specific version. If not specified, will be installed depending on the php version.

```

<!--ENDROLEVARS-->
6 changes: 3 additions & 3 deletions roles/deploy_code/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ With this method the live code directory is also the build directory, therefore
## `squashfs` builds
Because `tarball` is very slow, we have a second method using [`squashfs`](https://github.com/plougher/squashfs-tools). This filesystem is designed for packing and compressing files into read-only images - initially to deploy to removable media - that can simply be mounted, similar to a macOS Apple Disk Image (DWG) file. It is both faster to pack than a tarball *and* instant to deploy (it's just a `mount` command).

However, the build process is more complex. Because mounted `squashfs` images are read only, we cannot build over them as we do in other types of build. [We alter the build path variables in the `_init` role](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/_init/tasks/main.yml#L25) so the build happens in a separate place and then in the `cleanup.yml` we pack the built code into an image ready to be deployed. Again, because the images are read-only mounts, the live site needs to be *unmounted* with an `umount` command and then remounted with a `mount` command to be completely deployed. This requires the `ce-deploy` user to have extra `sudo` permissions, which is handled by [the `squashfs` role in `ce-provision`](https://github.com/codeenigma/ce-provision/blob/1.x/roles/squashfs)
However, the build process is more complex. Because mounted `squashfs` images are read only, we cannot build over them as we do in other types of build. [We alter the build path variables in the `_init` role](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/_init/tasks/main.yml#L25) so the build happens in a separate place and then in the `cleanup.yml` we pack the built code into an image ready to be deployed. Again, because the images are read-only mounts, the live site needs to be *unmounted* with an `umount` command and then remounted with a `mount` command to be completely deployed. This requires the `ce-deploy` user to have extra `sudo` permissions, which is handled by [the `squashfs` role in `ce-provision`](https://github.com/codeenigma/ce-provision/tree/1.x/roles/squashfs).

Consequently, at the build stage there are two important extra variables to set:

Expand All @@ -63,10 +63,10 @@ deploy_code:

`service_action` is whether `ce-deploy` should restart the services in the list of stop them, unmount and remount the image and start them again. The latter is the only "safe" way to deploy, but results in a second or two of down time.

Finally, as with the `tarball` method, the packed image is copied up to the NAS to be available to all future servers and is always named `deploy.sqsh`. The previous codebase is *also* packed and copied to the NAS, named `deploy_previous.sqsh` in the same directory.
Finally, as with the `tarball` method, the packed image is copied up to the NAS to be available to all future servers and is named `PROJECTNAME_BUILDTYPE_deploy.sqsh`. [See the code here.](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/deploy_code/tasks/cleanup.yml#L80) The previous codebase is *also* packed and copied to the NAS, named `PROJECTNAME_BUILDTYPE_deploy_previous.sqsh` in the same directory.

### Rolling back
Rolling back from a bad `squashfs` build means copying `deploy_previous.sqsh` down from the NAS to a sensible location in the `ce-deploy` user's home directory, unmounting the current image and mounting `deploy_previous.sqsh` in its place. Once you've done that, to ensure future autoscaling events do not load the bad code, on the NAS you will need to rename `deploy.sqsh` to something else (or delete it entirely if you're sure you don't want it) and rename `deploy_previous.sqsh` as `deploy.sqsh`, so it is used on an autoscale event.
Rolling back from a bad `squashfs` build means copying the project/branch specific `deploy_previous.sqsh` down from the NAS to a sensible location in the `ce-deploy` user's home directory, unmounting the current image and mounting the `deploy_previous.sqsh` in its place. Once you've done that, to ensure future autoscaling events do not load the bad code, on the NAS you will need to rename `deploy.sqsh` to something else (or delete it entirely if you're sure you don't want it) and rename your `deploy_previous.sqsh` as `PROJECTNAME_BUILDTYPE_deploy.sqsh`, so it is used on an autoscale event. [You can see the actual code that runs on autoscale events here.](https://github.com/codeenigma/ce-provision/blob/1.x/roles/mount_sync/templates/init-squashfs.sh.j2)

Same as with the `tarball` method, as long as the `database_backup` is using the `rolling` method then the "roll back" database will still exist and the credentials will be correct in the `deploy_previous.sqsh` image. Again, if the backup method is `dump` then you will need to inspect [the `mysql_backup.dumps_directory` variable](https://github.com/codeenigma/ce-deploy/blob/1.x/roles/database_backup/database_backup-mysql/defaults/main.yml#L4) to see where the backup was saved in order to restore it.

Expand Down
13 changes: 8 additions & 5 deletions roles/sanitize/admin_creds/admin_creds-drupal7/tasks/admin.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
---

- set_fact:
- ansible.builtin.set_fact:
_admin_user: "{{ lookup('password', '/tmp/{{ project_name }}-{{ site.folder }}-{{ build_type }}-{{ build_number }}-user chars=ascii_letters') }}"
- set_fact:
- ansible.builtin.set_fact:
_admin_pwd: "{{ lookup('password', '/tmp/{{ project_name }}-{{ site.folder }}-{{ build_type }}-{{ build_number }}-pwd chars=ascii_letters') }}"

- name: Reset admin username.
shell: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drush_bin }} -l {{ site.folder }} sql-query \"UPDATE users SET name='{{ _admin_user }}' WHERE uid=1;\""
ansible.builtin.command:
cmd: "{{ drush_bin }} -l {{ site.folder }} sql-query \"UPDATE users SET name='{{ _admin_user }}' WHERE uid=1;\""
chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
- name: Reset admin password.
shell: "cd {{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }} && {{ drush_bin }} -l {{ site.folder }} upwd {{ _admin_user }} --password='{{ _admin_pwd }}'"
ansible.builtin.command:
cmd: "{{ drush_bin }} -l {{ site.folder }} upwd {{ _admin_user }} --password='{{ _admin_pwd }}'"
chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
---
# For some unknown reason, updating the field_user_data table did not work reliably,
# nor did drush upwd for the password.
# Loading the user directly is akward, but at least means we don't bypass entity update.
# Loading the user directly is awkward, but at least means we don't bypass entity update.
- name: Reset admin username.
command:
ansible.builtin.command:
cmd: "{{ drush_bin }} -l {{ site.folder }} php-eval '$admin = \\Drupal\\user\\Entity\\User::load(1); $admin->setUsername(\"{{ admin_creds.username }}\");$admin->save();' "
chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
when:
- admin_creds.username | length > 1

- name: Reset admin password.
command:
ansible.builtin.command:
cmd: "{{ drush_bin }} -l {{ site.folder }} php-eval '$admin = \\Drupal\\user\\Entity\\User::load(1); $admin->setPassword(\"{{ admin_creds.password }}\");$admin->save();' "
chdir: "{{ deploy_path }}/{{ webroot }}/sites/{{ site.folder }}"
when:
Expand Down