0.9.0: Pull gpg homedir creation out of the main scriptworker process.

CHANGELOG.md

@@ -3,6 +3,25 @@ All notable changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/).
 
 ## Unreleased
+### Added
+- `gpg_lockfile` and `last_good_git_revision_file` in config
+- `get_last_good_git_revision` and `write_last_good_git_revision` now return the last good git revision, and write it to `last_good_git_revision_file`, respectively.
+- `get_tmp_base_gpg_home_dir` is a helper function to avoid duplication in logic.
+- `rebuild_gpg_homedirs` is a new entry point script that allows us to recreate the gpg homedirs in a tmpdir, in a separate process
+- `check_lockfile`, `create_lockfile`, and `rm_lockfile` as helper functions for the two gpg homedir entry points.
+
+### Changed
+- `sign_key`, `rebuild_gpg_home_flat`, `rebuild_gpg_home_signed`, `build_gpg_homedirs_from_repo` are no longer async.
+- `overwrite_gpg_home` only keeps one backup.
+- `update_signed_git_repo` now returns the latest git revision, instead of a boolean marking whether the revision is new or not.  This will help avoid the scenario where we update, fail to generate the gpg homedirs, and then stay on an old revision until the next push.
+- `update_logging_config` now takes a `file_name` kwarg, which allows us to create new log files for the `rebuild_gpg_homedirs` and `create_initial_gpg_homedirs` entry points.
+
+### Fixed
+- `build_gpg_homedirs_from_repo` now waits to verify the contents of the updated git repo before nuking the previous base gpg homedir.
+- `create_initial_gpg_homedirs` now creates a logfile
+
+### Removed
+- `rebuild_gpg_homedirs_loop` is no longer needed, and is removed.
 
 ## [0.8.2] - 2016-10-24
 ### Changed

config_example.json

@@ -10,6 +10,8 @@
     "task_log_dir": "/tmp/artifact/public/logs",
     "git_key_repo_dir": "/tmp/key_repo",
     "base_gpg_home_dir": "/tmp/gpg",
+    "gpg_lockfile": "/tmp/gpg_homedir.lock",
+    "last_good_git_revision_file": "/tmp/git_revision",
 
     "my_email": "scriptworker@example.com",
 

scriptworker/constants.py

@@ -62,20 +62,23 @@
     # Boolean to use the gpg agent
     "gpg_use_agent": False,
     "gpg_encoding": 'utf-8',
+    "gpg_lockfile": os.path.join(os.getcwd(), "gpg_homedir.lock"),
 
     # Worker log settings
     "log_datefmt": "%Y-%m-%dT%H:%M:%S",
     "log_fmt": "%(asctime)s %(levelname)8s - %(message)s",
     "log_max_bytes": 1024 * 1024 * 512,
     "log_num_backups": 10,
 
+    "git_key_repo_dir": "...",
+    "base_gpg_home_dir": "...",
+    "last_good_git_revision_file": os.path.join(os.getcwd(), "git_revision"),
+
     # Task settings
     "work_dir": "...",
     "log_dir": "...",
     "artifact_dir": "...",
     "task_log_dir": "...",  # set this to ARTIFACT_DIR/public/logs
-    "git_key_repo_dir": "...",
-    "base_gpg_home_dir": "...",
     "artifact_expiration_hours": 24,
     "artifact_upload_timeout": 60 * 20,
     "sign_key_timeout": 60 * 2,