diff --git a/Berksfile b/Berksfile new file mode 100644 index 0000000..b63d34a --- /dev/null +++ b/Berksfile @@ -0,0 +1,3 @@ +source 'https://supermarket.chef.io' + +cookbook 'apt' \ No newline at end of file diff --git a/Berksfile.lock b/Berksfile.lock new file mode 100644 index 0000000..243037f --- /dev/null +++ b/Berksfile.lock @@ -0,0 +1,5 @@ +DEPENDENCIES + apt + +GRAPH + apt (7.2.0) diff --git a/README.md b/README.md new file mode 100644 index 0000000..8cc8c48 --- /dev/null +++ b/README.md @@ -0,0 +1,98 @@ +# Pentest lab on GCP using Terraform, Chef, Ansible, and Vagrant + +_Because why not use all the things?_ + +This repo is a general stash of my setup for how I provisioned and managed a vpn private network, including a +to-be-assessed ultra-vulnerable server, for each of twenty teams in my information security management class, Fall 2019. + +I wanted to be able to borrow from and contribute back to [metasploitable3's ub1404 project](https://github.com/rapid7/metasploitable3), so I used chef for provisioning +the vulnerable server. I used vagrant to play the chef recipe provisioning on top of a `ubuntu-1404-lts` gcp box. When the midterm +server was mostly complete, I manually created a gcp image of it. I could have used [packer to create the image via chef-solo](https://www.packer.io/docs/provisioners/chef-solo.html). + +I used terraform to create a workspace for each team. Each workspace has: + +* an isolated vpc +* firewall rules allowing only ssh from the outside world, but allowing all traffic on the private network +* an openvpn server provisioned via a customized easy-openvpn script +* a copy of a vulnerable server based on the most recent image from my image family for the vuln server. + +I used a few bash scripts for automating the process of managing all 20 workspaces -- for `terraform apply`ing, etc. Naturally, I had to do +some hot fixes to the vulnerable servers in the middle of the assignment timeframe. While I could have updated the base vuln server image and +had terraform tear down and recreate each of the vuln servers for each workspace, this led to too much downtime for the students, and was too +tedious for me. So, I initially looked to using a Chef Server for managing each vuln server. But I gave up on this when I hit some gotchas with +trying to automate the `knife` provisioning of each vuln server with the Chef Server. I switched to ansible instead, dynamically creating +the ansible inventory file from a script iterating over `terraform output` which spit out each workspace's vuln server ip. The ansible +playbook uploads a zip of the within-project chef recipes, runs them on the server, and then deletes the zip. I'm sure there's a cleaner +way to do that, but hey, it's fast and easy, and it works. + +I'm removing most of my chef recipes from this public repo -- except for the basic functionality ones -- just to make it harder for nosy students to complete the midterm task. On the other hand, +heh, they're business school students, they probably can't read ruby anyhow. + + + +## GCP + +I created a gcp project specifically for this midterm assignment. +I also `ssh-keygen` created a private-public keypair specifically for provisioning for this project, +and I added this public key to the gcp project metadata, with username `_provisioner`. In the files in this +repo, I also added this private-public keypair as user `tyler`'s. That's what the reference to `tyler-midterm-vuln` is +in some of the config files. It's used to connect as the `_provisioner`. + +I also created a gcp service account for this project (can't remember what permissions I gave it), and I downloaded it as json +and called it `midterm-vuln-gcp-private-key.json`. + + +## Vagrant and Chef + +Run these: + + vagrant plugin install vagrant-vbguest + vagrant plugin install vagrant-google + +The chef-solo runlist is read from a file `chef_runlist`, because the ansible playbook also needs it. + +Do your initial vuln midterm server creation using `vagrant up`, `vagrant provision`, etc. When you're satisfied, create a gcp image +from your instance. For compatibility with this repo's terraform config (`main.tf`), call your image family `midterm-vuln`. + +I managed external chef cookbook dependencies using `berks`, which is provided by the `ChefDK`. Running `berks vendor` puts the berks cookbooks in a separate directory (IIRC). + + + +## Terraform + +I named my teams like this: `team- 1 ? local.team_number_split[1] : 1 + google_region = local.gcp_regions[(local.team_number - 1) % length(local.gcp_regions)] + } + + +My vuln midterm server "company" was called `humbleify` (`resource "google_compute_instance" "midterm-vuln"`). The humbleify server was given an ip address of `192.168.10.107`. + +The vpn server was provisioned via (`resource "google_compute_instance" "openvpn"`). It is based on gcp image debian-9. It is provisioned (`resource "null_resource" "openvpn_bootstrap"`) via `openvpn-install.sh`, +which is a fork of `https://raw.githubusercontent.com/angristan/openvpn-install/master/openvpn-install.sh`. This vpn server has some crucial key differences from one used solely as a tunnel for an +individual computer's traffic: + +* It allows clients on the vpn network to see and talk to one another +* It allows multiple clients to simultaneously use the same config file. + +After the vpn server is bootstrapped, the `client.conf` config file is `scp`'ed down to `vpn_configs/`, and then uploaded to the vuln midterm server (`resource "null_resource" "openvpn_upload_midtermvuln_vpn_config"). +I had already installed the openvpn package on the midterm servers in my base image. + + +## Ansible hot-patching + +I dynamically managed the inventory by running `scripts/get_all_ips.sh`, which relies on `output "midterm_vuln_ip"` and `output "team-name"` being provided by the terraform workspaces. + +I created a `team-999` workspace that I used for testing my chef recipe live provision hot-fixes. `scripts/ansible-playbook-999.sh` runs the ansible playbook for just `team-999`. +After the success of the hot-patch was confirmed, I ran `scripts/ansible-playbook.sh`. + + diff --git a/Vagrantfile b/Vagrantfile new file mode 100644 index 0000000..806995b --- /dev/null +++ b/Vagrantfile @@ -0,0 +1,48 @@ +# -*- mode: ruby -*- +# vi: set ft=ruby : + + +# All Vagrant configuration is done below. The "2" in Vagrant.configure +# configures the configuration version (we support older styles for +# backwards compatibility). Please don't change it unless you know what +# you're doing. +Vagrant.configure("2") do |config| + # Vagrant::DEFAULT_SERVER_URL.replace('https://vagrantcloud.com') + # The most common configuration options are documented and commented below. + # For a complete reference, please see the online documentation at + # https://docs.vagrantup.com. + + # Every Vagrant development environment requires a box. You can search for + # boxes at https://vagrantcloud.com/search. + + config.vm.define "gcp" do |gcp| + gcp.vm.box = 'google/gce' + gcp.vm.box_version = "0.1.0" + + gcp.vm.provider :google do |google, override| + google.google_project_id = "midterm-vuln" + google.name = 'midterm-vuln' + google.google_json_key_location = "midterm-vuln-gcp-private-key.json" + + # google.external_ip = true + google.image_family = 'ubuntu-1404-lts' + # vagrant@terraform-255421.iam.gserviceaccount.com + override.ssh.username = "_provisioner" + override.ssh.private_key_path = "tyler-midterm-vuln" + end + end + + + # Disable automatic box update checking. If you disable this, then + # boxes will only be checked for updates when the user runs + # `vagrant box outdated`. This is not recommended. + # config.vm.box_check_update = false + # config.vm.synced_folder '.', '/vagrant', disabled: true + + config.vm.provision "chef_solo" do |chef| + chef.arguments = '--chef-license accept' + chef.cookbooks_path = ['cookbooks','berks-cookbooks'] + run_list = File.read('chef_runlist') + end + +end diff --git a/ansible.cfg b/ansible.cfg new file mode 100644 index 0000000..14c8065 --- /dev/null +++ b/ansible.cfg @@ -0,0 +1,2 @@ +[defaults] +host_key_checking = False diff --git a/berks-cookbooks/apt/CHANGELOG.md b/berks-cookbooks/apt/CHANGELOG.md new file mode 100644 index 0000000..6b9b2b4 --- /dev/null +++ b/berks-cookbooks/apt/CHANGELOG.md @@ -0,0 +1,368 @@ +# apt Cookbook CHANGELOG + +This file is used to list changes made in each version of the apt cookbook. + +## 7.2.0 (2019-08-05) + +- Allow you to specify dpkg options just for unattended upgrades - [@majormoses](https://github.com/majormoses) +- Adding documentation and tests for setting dpkg options unattended upgrades - [@majormoses](https://github.com/majormoses) +- Test on Chef 15 + Chef Workstation - [@tas50](https://github.com/tas50) +- Remove tests of the resources now built into Chef - [@tas50](https://github.com/tas50) +- Remove respond_to from the metadata - [@tas50](https://github.com/tas50) +- Remove the recipe description from the metadata as these aren't used - [@tas50](https://github.com/tas50) +- Replace Chef 12 testing with 13.3 - [@tas50](https://github.com/tas50) +- Remove Ubuntu 14.04 / Debian 8 testing and add Debian 10 testing - [@tas50](https://github.com/tas50) + +## 7.1.1 (2018-10-11) + +- Allow to customize sender email for unattended-upgrades + +## 7.1.0 (2018-09-05) + +- Add the installation of dirmngr and gnupg to the apt default cookbook to support secure repositories +- Added support for the unattended-upgrade SyslogEnable configuration feature +- Added support for the unattended-upgrade SyslogFacility configuration feature + +## 7.0.0 (2018-04-06) + +### Breaking Change + +- This cookbook no longer includes apt_preference as that resource was moved into Chef Client 13.3. The cookbook now also requires Chef 13.3 or later. If you require support for an older release of Chef you will need to pin to a 6.X release. + +## 6.1.4 (2017-08-31) + +- Restores ignore_failure true on compile time update. +- name_property vs name_attribute in the resource + +## 6.1.3 (2017-07-19) + +- Fixed typo in readme +- Fixed config namespace in the 10dpkg-options file + +## 6.1.2 (2017-06-20) + +- restore backwards compatability by respecting node['apt']['periodic_update_min_delay'] + +## 6.1.1 (2017-06-20) + +- Remove action_class.class_eval usage that caused failures +- Remove wrong warning logs generated by apt_preference +- Fix wrong warning log in cacher-client recipe + +## 6.1.0 (2017-04-11) + +- Test with local delivery and not Rake +- Use proper value type for bsd-mailx package only_if/not_if block +- Update apache2 license string +- Convert apt_preference to a custom resource + +## 6.0.1 (2017-02-27) + +- Update cookbook description +- Testing updates for Chef 13 and fixes to the cacher recipe + +## 6.0.0 (2017-02-08) + +### Breaking changes + +- apt_update and apt_repository resources have been removed from the cookbook. These resources were both added to the chef-client itself. Due to this we now require Chef 12.9 or later, which has both of these resources built in. If you require compatibility with older chef-client releases you will need to pin to the 5.X release. + +### Other changes + +- apt_preference resource now properly required a pin_priority, which prevents us from writing out bad preference files that must be manually removed + +## 5.1.0 (2017-02-01) + +- Convert integration tests to inspec +- Add management of the /etc/apt/apt.conf.d/10dpkg-options file with new attributes. This allows tuning of how dpkg will handle package prompts during package installation. Note that Chef 12.19+ will automatically suppress package prompts + +## 5.0.1 (2016-12-22) + +- Avoid CHEF-3694 in apt_preferences resource +- Cookstyle fixes + +## 5.0.0 (2016-10-14) + +- Remove search logic from the cacher client cookbook and rework attribute structure. See the attributes file and readme for new structure. Determining what servers to cache against is better handled in a wrapper cookbook where you can define the exact search syntax yourself +- Corrected readme examples for the cacher client setup +- Depend on the latest compat_resource +- Define matchers for ChefSpec +- Testing updates to better test the various recipes and providers in the cookbook on Travis + +## 4.0.2 (2016-08-13) + +- The cookbook requires Chef 12.1+ not 12.0\. Update docs +- Test on Chef 12.1 to ensure compatibility +- Restore compatibility with Chef < 12.4 + +## 4.0.1 (2016-06-21) + +- Fix bug that prevented adding the cookbook to non Debian/Ubuntu nodes without chef run failures + +## 4.0.0 (2016-06-02) + +This cookbook now requires Chef 12\. If you require Chef 11 compatibility you will need to pin to the 3.X cookbook version + +- The apt-get update logic in the default recipe has been converted to apt_update custom resource and compat_resource cookbook has been added for backwards compatibility with all Chef 12.X releases. In addition this resource is now included in core chef-client and the cookbook will use the built-in resource if available +- Added support for the unattended-upgrade RandomSleep configuration feature +- Added support for the unattended-upgrade Unattended-Upgrade::Origins-Pattern configuration feature +- Added Chefspec matchers for apt_update +- Fixed apt_repository documentation to correctly reflect the deb_src property + +## 3.0.0 (2016-03-01) + +- Removed Chef 10 compatibility code. This cookbook requires Chef 11 or greater now +- The default recipe will no longer create /etc/apt/ and other directories on non-Debian based systems +- Updated the autoremove command in the default recipe to run in non-interactive mode +- Added CentOS 7 to Test Kitchenwith tests to ensure we don't create any files on RHEL or other non-Debian hosts +- Updated Chefspec to 4.X format +- Properly mock the existence of apt for the Chefspec runs so they don't just skip over the resources +- Fixed lwrp test kitchen tests to pass +- Resolved or disabled all Rubocop warnings +- Enabled testing in Travis CI +- Removed Apt Cacher NG support for Ubuntu 10.04 and Debian 6.X as they are both deprecated +- Fixed + signs in packages names with the preference LWRP being rejected + +## v2.9.2 + +- # 168 Adding guard to package resource. + +## v2.9.1 + +- Adding package apt-transport-https to default.rb + +## v2.9.0 + +- Add `sensitive` flag for apt_repositories +- Enable installation of recommended or suggested packages +- Tidy up `apt-get update` logic +- Fixing not_if guard on ruby_block[validate-key #{key}] + +## v2.8.2 (2015-08-24) + +- Fix removal of apt_preferences + +## v2.8.1 (2015-08-18) + +- Handle keyservers as URLs and bare hostnames + +## v2.8.0 (2015-08-18) + +- Access keyservers on port 80 +- Adds key_proxy as LWRP attribute for apt_repository +- Fix wildcard glob preferences files +- Fix text output verification for non en_US locales +- Quote repo URLs to deal with spaces + +## v2.7.0 (2015-03-23) + +- Support Debian 8.0 +- Filename verification for LWRPs +- Support SSL enabled apt repositories + +## v2.6.1 (2014-12-29) + +- Remove old preference files without .pref extension from previous versions + +## v2.6.0 (2014-09-09) + +- Always update on first run - check +- Adding ppa support for apt_repository + +## v2.5.3 (2014-08-14) + +- # 87 - Improve default settings, account for non-linux platforms + +## v2.5.2 (2014-08-14) + +- Fully restore 2.3.10 behaviour + +## v2.5.1 (2014-08-14) + +- fix breakage introduced in apt 2.5.0 + +## v2.5.0 (2014-08-12) + +- Add unattended-upgrades recipe +- Only update the cache for the created repository +- Added ChefSpec matchers and default_action for resources +- Avoid cloning resource attributes +- Minor documentation updates + +## v2.4.0 (2014-05-15) + +- [COOK-4534]: Add option to update apt cache at compile time + +## v2.3.10 (2014-04-23) + +- [COOK-4512] Bugfix: Use empty PATH if PATH is nil + +## v2.3.8 (2014-02-14) + +### Bug + +- **[COOK-4287](https://tickets.opscode.com/browse/COOK-4287)** - Cleanup the Kitchen + +## v2.3.6 + +- [COOK-4154] - Add chefspec matchers.rb file to apt cookbook +- [COOK-4102] - Only index created repository + +## v2.3.6 + +- [COOK-4154] - Add chefspec matchers.rb file to apt cookbook +- [COOK-4102] - Only index created repository + +## v2.3.4 + +No change. Version bump for toolchain sanity + +## v2.3.2 + +- [COOK-3905] apt-get-update-periodic: configuration for the update period +- Updating style for rubocops +- Updating test-kitchen harness + +## v2.3.0 + +### Bug + +- **[COOK-3812](https://tickets.opscode.com/browse/COOK-3812)** - Add a way to bypass the apt existence check + +### Improvement + +- **[COOK-3567](https://tickets.opscode.com/browse/COOK-3567)** - Allow users to bypass apt-cache via attributes + +## v2.2.1 + +### Improvement + +- **[COOK-664](https://tickets.opscode.com/browse/COOK-664)** - Check platform before running apt-specific commands + +## v2.2.0 + +### Bug + +- **[COOK-3707](https://tickets.opscode.com/browse/COOK-3707)** - multiple nics confuse apt::cacher-client + +## v2.1.2 + +### Improvement + +- **[COOK-3551](https://tickets.opscode.com/browse/COOK-3551)** - Allow user to set up a trusted APT repository + +## v2.1.1 + +### Bug + +- **[COOK-1856](https://tickets.opscode.com/browse/COOK-1856)** - Match GPG keys without case sensitivity + +## v2.1.0 + +- [COOK-3426]: cacher-ng fails with restrict_environment set to true +- [COOK-2859]: cacher-client executes out of order +- [COOK-3052]: Long GPG keys are downloaded on every run +- [COOK-1856]: apt cookbook should match keys without case sensitivity +- [COOK-3255]: Attribute name incorrect in README +- [COOK-3225]: Call use_inline_resources only if defined +- [COOK-3386]: Cache dir for apt-cacher-ng +- [COOK-3291]: apt_repository: enable usage of a keyserver on port 80 +- Greatly expanded test coverage with ChefSpec and Test-Kitchen + +## v2.0.0 + +### Bug + +- [COOK-2258]: apt: LWRP results in error under why-run mode in apt 1.9.0 cookbook + +## v1.10.0 + +### Improvement + +- [COOK-2885]: Improvements for apt cache server search + +### Bug + +- [COOK-2441]: Apt recipe broken in new chef version +- [COOK-2660]: Create Debian 6.0 "squeeze" specific template for +- apt-cacher-ng + +## v1.9.2 + +- [COOK-2631] - Create Ubuntu 10.04 specific template for apt-cacher-ng + +## v1.9.0 + +- [COOK-2185] - Proxy for apt-key +- [COOK-2338] - Support pinning by glob() or regexp + +## v1.8.4 + +- [COOK-2171] - Update README to clarify required Chef version: 10.18.0 +- or higher. + +## v1.8.2 + +- [COOK-2112] - need [] around "arch" in sources.list entries +- [COOK-2171] - fixes a regression in the notification + +## v1.8.0 + +- [COOK-2143] - Allow for a custom cacher-ng port +- [COOK-2171] - On `apt_repository.run_action(:add)` the source file +- is not created. +- [COOK-2184] - apt::cacher-ng, use `cacher_port` attribute in +- acng.conf + +## v1.7.0 + +- [COOK-2082] - add "arch" parameter to apt_repository LWRP + +## v1.6.0 + +- [COOK-1893] - `apt_preference` use "`package_name`" resource instead of "name" +- [COOK-1894] - change filename for sources.list.d files +- [COOK-1914] - Wrong dir permissions for /etc/apt/preferences.d/ +- [COOK-1942] - README.md has wrong name for the keyserver attribute +- [COOK-2019] - create 01proxy before any other apt-get updates get executed + +## v1.5.2 + +- [COOK-1682] - use template instead of file resource in apt::cacher-client +- [COOK-1875] - cacher-client should be Environment-aware + +## V1.5.0 + +- [COOK-1500] - Avoid triggering apt-get update +- [COOK-1548] - Add execute commands for autoclean and autoremove +- [COOK-1591] - Setting up the apt proxy should leave https +- connections direct +- [COOK-1596] - execute[apt-get-update-periodic] never runs +- [COOK-1762] - create /etc/apt/preferences.d directory +- [COOK-1776] - apt key check isn't idempotent + +## v1.4.8 + +- Adds test-kitchen support +- [COOK-1435] - repository lwrp is not idempotent with http key + +## v1.4.6 + +- [COOK-1530] - apt_repository isn't aware of update-success-stamp +- file (also reverts COOK-1382 patch). + +## v1.4.4 + +- [COOK-1229] - Allow cacher IP to be set manually in non-Chef Solo +- environments +- [COOK-1530] - Immediately update apt-cache when sources.list file is dropped off + +## v1.4.2 + +- [COOK-1155] - LWRP for apt pinning + +## v1.4.0 + +- [COOK-889] - overwrite existing repo source files +- [COOK-921] - optionally use cookbook_file or remote_file for key +- [COOK-1032] - fixes problem with apt repository key installation diff --git a/berks-cookbooks/apt/CONTRIBUTING.md b/berks-cookbooks/apt/CONTRIBUTING.md new file mode 100644 index 0000000..ef2f2b8 --- /dev/null +++ b/berks-cookbooks/apt/CONTRIBUTING.md @@ -0,0 +1,2 @@ +Please refer to +https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD diff --git a/berks-cookbooks/apt/README.md b/berks-cookbooks/apt/README.md new file mode 100644 index 0000000..d54e58a --- /dev/null +++ b/berks-cookbooks/apt/README.md @@ -0,0 +1,215 @@ +# apt Cookbook + +[![Build Status](https://img.shields.io/travis/chef-cookbooks/apt.svg)][travis] [![Cookbook Version](https://img.shields.io/cookbook/v/apt.svg)][cookbook] + +This cookbook includes recipes to execute apt-get update to ensure the local APT package cache is up to date. There are recipes for managing the apt-cacher-ng caching proxy and proxy clients. It also includes a custom resource for pinning packages via /etc/apt/preferences.d. + +## Requirements + +### Platforms + +- Ubuntu 12.04+ +- Debian 7+ + +May work with or without modification on other Debian derivatives. + +### Chef + +- Chef 13.3+ + +### Cookbooks + +- None + +## Recipes + +### default + +This recipe manually updates the timestamp file used to only run `apt-get update` if the cache is more than one day old. + +This recipe should appear first in the run list of Debian or Ubuntu nodes to ensure that the package cache is up to date before managing any `package` resources with Chef. + +This recipe also sets up a local cache directory for preseeding packages. + +**Including the default recipe on a node that does not support apt (such as Windows or RHEL) results in a noop.** + +### cacher-client + +Configures the node to use a `apt-cacher-ng` server to cache apt requests. Configuration of the server to use is located in `default['apt']['cacher_client']['cacher_server']` which is a hash containing `host`, `port`, `proxy_ssl`, and `bypass` keys. Example: + +```json +{ + "apt": { + "cacher_client": { + "cacher_server": { + "host": "cache_server.mycorp.dmz", + "port": 1234, + "proxy_ssl": true, + "cache_bypass": { + "download.oracle.com": "http" + } + } + } + } +} +``` + +#### Bypassing the cache + +Occasionally you may come across repositories that do not play nicely when the node is using an `apt-cacher-ng` server. You can configure `cacher-client` to bypass the server and connect directly to the repository with the `cache_bypass` attribute. + +To do this, you need to override the `cache_bypass` attribute with an hash of repositories, with each key as the repository URL and value as the protocol to use: + +```json +{ + "apt": { + "cacher_client": { + "cacher_server": { + "cache_bypass": { + "URL": "PROTOCOL" + } + } + } + } +} +``` + +For example, to prevent caching and directly connect to the repository at `download.oracle.com` via http and the repo at `nginx.org` via https + +```json +{ + "apt": { + "cacher_client": { + "cacher_server": { + "cache_bypass": { + "download.oracle.com": "http", + "nginx.org": "https" + } + } + } + } +} +``` + +### cacher-ng + +Installs the `apt-cacher-ng` package and service so the system can provide APT caching. You can check the usage report at . + +If you wish to help the `cacher-ng` recipe seed itself, you must now explicitly include the `cacher-client` recipe in your run list **after** `cacher-ng` or you will block your ability to install any packages (ie. `apt-cacher-ng`). + +### unattended-upgrades + +Installs and configures the `unattended-upgrades` package to provide automatic package updates. This can be configured to upgrade all packages or to just install security updates by setting `['apt']['unattended_upgrades']['allowed_origins']`. + +To pull just security updates, set `origins_patterns` to something like `["origin=Ubuntu,archive=trusty-security"]` (for Ubuntu trusty) or `["origin=Debian,label=Debian-Security"]` (for Debian). + +## Attributes + +### General + +- `['apt']['compile_time_update']` - force the default recipe to run `apt-get update` at compile time. +- `['apt']['periodic_update_min_delay']` - minimum delay (in seconds) between two actual executions of `apt-get update` by the `execute[apt-get-update-periodic]` resource, default is '86400' (24 hours) + +### Caching + +- `['apt']['cacher_client']['cacher_server']` - Hash containing server information used by clients for caching. See the example in the recipes section above for the full format of the hash. +- `['apt']['cacher_interface']` - interface to connect to the cacher-ng service, no default. +- `['apt']['cacher_port']` - port for the cacher-ng service (used by server recipe only), default is '3142' +- `['apt']['cacher_dir']` - directory used by cacher-ng service, default is '/var/cache/apt-cacher-ng' +- `['apt']['compiletime']` - force the `cacher-client` recipe to run before other recipes. It forces apt to use the proxy before other recipes run. Useful if your nodes have limited access to public apt repositories. This is overridden if the `cacher-ng` recipe is in your run list. Default is 'false' + +### Unattended Upgrades + +- `['apt']['unattended_upgrades']['enable']` - enables unattended upgrades, default is false +- `['apt']['unattended_upgrades']['update_package_lists']` - automatically update package list (`apt-get update`) daily, default is true +- `['apt']['unattended_upgrades']['allowed_origins']` - array of allowed apt origins from which to pull automatic upgrades, defaults to a guess at the system's main origin and should almost always be overridden +- `['apt']['unattended_upgrades']['origins_patterns']` - array of allowed apt origin patterns from which to pull automatic upgrades, defaults to none. +- `['apt']['unattended_upgrades']['package_blacklist']` - an array of package which should never be automatically upgraded, defaults to none +- `['apt']['unattended_upgrades']['auto_fix_interrupted_dpkg']` - attempts to repair dpkg state with `dpkg --force-confold --configure -a` if it exits uncleanly, defaults to false (contrary to the unattended-upgrades default) +- `['apt']['unattended_upgrades']['minimal_steps']` - Split the upgrade into the smallest possible chunks. This makes the upgrade a bit slower but it has the benefit that shutdown while a upgrade is running is possible (with a small delay). Defaults to false. +- `['apt']['unattended_upgrades']['install_on_shutdown']` - Install upgrades when the machine is shuting down instead of doing it in the background while the machine is running. This will (obviously) make shutdown slower. Defaults to false. +- `['apt']['unattended_upgrades']['mail']` - Send email to this address for problems or packages upgrades. Defaults to no email. +- `['apt']['unattended_upgrades']['sender']` - Send email from this address for problems or packages upgrades. Defaults to 'root'. +- `['apt']['unattended_upgrades']['mail_only_on_error']` - If set, email will only be set on upgrade errors. Otherwise, an email will be sent after each upgrade. Defaults to true. +- `['apt']['unattended_upgrades']['remove_unused_dependencies']` Do automatic removal of new unused dependencies after the upgrade. Defaults to false. +- `['apt']['unattended_upgrades']['automatic_reboot']` - Automatically reboots _without confirmation_ if a restart is required after the upgrade. Defaults to false. +- `['apt']['unattended_upgrades']['dl_limit']` - Limits the bandwidth used by apt to download packages. Value given as an integer in kb/sec. Defaults to nil (no limit). +- `['apt']['unattended_upgrades']['random_sleep']` - Wait a random number of seconds up to this value before running daily periodic apt actions. System default is 1800 seconds (30 minutes). +- `['apt']['unattended_upgrades']['syslog_enable']` - Enable logging to syslog. Defaults to false. +- `['apt']['unattended_upgrades']['syslog_facility']` - Specify syslog facility. Defaults to 'daemon'. +- `['apt']['unattended_upgrades']['dpkg_options']` An array of dpkg options to be used specifically only for unattended upgrades. Defaults to `[]` which will prevent it from being rendered from the template in the resulting file. + +### Configuration for APT + +- `['apt']['confd']['force_confask']` - Prompt when overwriting configuration files. (default: false) +- `['apt']['confd']['force_confdef']` - Don't prompt when overwriting configuration files. (default: false) +- `['apt']['confd']['force_confmiss']` - Install removed configuration files when upgrading packages. (default: false) +- `['apt']['confd']['force_confnew']` - Overwrite configuration files when installing packages. (default: false) +- `['apt']['confd']['force_confold']` - Keep modified configuration files when installing packages. (default: false) +- `['apt']['confd']['install_recommends']` - Consider recommended packages as a dependency for installing. (default: true) +- `['apt']['confd']['install_suggests']` - Consider suggested packages as a dependency for installing. (default: false) + +## Libraries + +There is an `interface_ipaddress` method that returns the IP address for a particular host and interface, used by the `cacher-client` recipe. To enable it on the server use the `['apt']['cacher_interface']` attribute. + +## Usage + +Put `recipe[apt]` first in the run list. If you have other recipes that you want to use to configure how apt behaves, like new sources, notify the execute resource to run, e.g.: + +```ruby +template '/etc/apt/sources.list.d/my_apt_sources.list' do + notifies :run, 'execute[apt-get update]', :immediately +end +``` + +The above will run during execution phase since it is a normal template resource, and should appear before other package resources that need the sources in the template. + +Put `recipe[apt::cacher-ng]` in the run_list for a server to provide APT caching and add `recipe[apt::cacher-client]` on the rest of the Debian-based nodes to take advantage of the caching server. + +If you want to cleanup unused packages, there is also the `apt-get autoclean` and `apt-get autoremove` resources provided for automated cleanup. + +## Resources + +### apt_preference + +The apt_preference resource has been moved into chef-client in Chef 13.3. + +See for usage details + +### apt_repository + +The apt_repository resource has been moved into chef-client in Chef 12.9. + +See for usage details + +### apt_update + +The apt_update resource has been moved into chef-client in Chef 12.7. + +See for usage details + +## Maintainers + +This cookbook is maintained by Chef's Community Cookbook Engineering team. Our goal is to improve cookbook quality and to aid the community in contributing to cookbooks. To learn more about our team, process, and design goals see our [team documentation](https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/COOKBOOK_TEAM.MD). To learn more about contributing to cookbooks like this see our [contributing documentation](https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD), or if you have general questions about this cookbook come chat with us in #cookbok-engineering on the [Chef Community Slack](http://community-slack.chef.io/) + +## License + +**Copyright:** 2009-2017, Chef Software, Inc. + +``` +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +``` + +[cookbook]: https://community.chef.io/cookbooks/apt +[travis]: https://travis-ci.org/chef-cookbooks/apt diff --git a/berks-cookbooks/apt/attributes/default.rb b/berks-cookbooks/apt/attributes/default.rb new file mode 100644 index 0000000..2d3e673 --- /dev/null +++ b/berks-cookbooks/apt/attributes/default.rb @@ -0,0 +1,62 @@ +# +# Cookbook:: apt +# Attributes:: default +# +# Copyright:: 2009-2017, Chef Software, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +default['apt']['cacher_dir'] = '/var/cache/apt-cacher-ng' +default['apt']['cacher_interface'] = nil +default['apt']['cacher_port'] = 3142 +default['apt']['compiletime'] = false +default['apt']['compile_time_update'] = false +default['apt']['key_proxy'] = '' +default['apt']['periodic_update_min_delay'] = 86_400 +default['apt']['launchpad_api_version'] = '1.0' +default['apt']['unattended_upgrades']['enable'] = false +default['apt']['unattended_upgrades']['update_package_lists'] = true +# this needs a good default +codename = node.attribute?('lsb') ? node['lsb']['codename'] : 'notlinux' +default['apt']['unattended_upgrades']['allowed_origins'] = [ + "#{node['platform'].capitalize} #{codename}", +] + +default['apt']['cacher_client']['cacher_server'] = {} + +default['apt']['unattended_upgrades']['origins_patterns'] = [] +default['apt']['unattended_upgrades']['package_blacklist'] = [] +default['apt']['unattended_upgrades']['auto_fix_interrupted_dpkg'] = false +default['apt']['unattended_upgrades']['minimal_steps'] = false +default['apt']['unattended_upgrades']['install_on_shutdown'] = false +default['apt']['unattended_upgrades']['mail'] = nil +default['apt']['unattended_upgrades']['sender'] = nil +default['apt']['unattended_upgrades']['mail_only_on_error'] = true +default['apt']['unattended_upgrades']['remove_unused_dependencies'] = false +default['apt']['unattended_upgrades']['automatic_reboot'] = false +default['apt']['unattended_upgrades']['automatic_reboot_time'] = 'now' +default['apt']['unattended_upgrades']['dl_limit'] = nil +default['apt']['unattended_upgrades']['random_sleep'] = nil +default['apt']['unattended_upgrades']['syslog_enable'] = false +default['apt']['unattended_upgrades']['syslog_facility'] = 'daemon' + +default['apt']['unattended_upgrades']['dpkg_options'] = [] + +default['apt']['confd']['force_confask'] = false +default['apt']['confd']['force_confdef'] = false +default['apt']['confd']['force_confmiss'] = false +default['apt']['confd']['force_confnew'] = false +default['apt']['confd']['force_confold'] = false +default['apt']['confd']['install_recommends'] = true +default['apt']['confd']['install_suggests'] = false diff --git a/berks-cookbooks/apt/files/15update-stamp b/berks-cookbooks/apt/files/15update-stamp new file mode 100644 index 0000000..14ead83 --- /dev/null +++ b/berks-cookbooks/apt/files/15update-stamp @@ -0,0 +1 @@ +APT::Update::Post-Invoke-Success {"touch /var/lib/apt/periodic/update-success-stamp 2>/dev/null || true";}; diff --git a/berks-cookbooks/apt/files/apt-proxy-v2.conf b/berks-cookbooks/apt/files/apt-proxy-v2.conf new file mode 100644 index 0000000..6954004 --- /dev/null +++ b/berks-cookbooks/apt/files/apt-proxy-v2.conf @@ -0,0 +1,50 @@ +[DEFAULT] +;; All times are in seconds, but you can add a suffix +;; for minutes(m), hours(h) or days(d) + +;; commented out address so apt-proxy will listen on all IPs +;; address = 127.0.0.1 +port = 9999 +cache_dir = /var/cache/apt-proxy + +;; Control files (Packages/Sources/Contents) refresh rate +min_refresh_delay = 1s +complete_clientless_downloads = 1 + +;; Debugging settings. +debug = all:4 db:0 + +time = 30 +passive_ftp = on + +;;-------------------------------------------------------------- +;; Cache housekeeping + +cleanup_freq = 1d +max_age = 120d +max_versions = 3 + +;;--------------------------------------------------------------- +;; Backend servers +;; +;; Place each server in its own [section] + +[ubuntu] +; Ubuntu archive +backends = + http://us.archive.ubuntu.com/ubuntu + +[ubuntu-security] +; Ubuntu security updates +backends = http://security.ubuntu.com/ubuntu + +[debian] +;; Backend servers, in order of preference +backends = + http://debian.osuosl.org/debian/ + +[security] +;; Debian security archive +backends = + http://security.debian.org/debian-security + http://ftp2.de.debian.org/debian-security diff --git a/berks-cookbooks/apt/libraries/helpers.rb b/berks-cookbooks/apt/libraries/helpers.rb new file mode 100644 index 0000000..6d98060 --- /dev/null +++ b/berks-cookbooks/apt/libraries/helpers.rb @@ -0,0 +1,49 @@ +# +# Cookbook:: apt +# Library:: helpers +# +# Copyright:: 2013-2017, Chef Software, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +module Apt + # Helpers for apt + module Helpers + # Determines if apt is installed on a system. + # + # @return [Boolean] + def apt_installed? + !which('apt-get').nil? + end + + # Finds a command in $PATH + # + # @return [String, nil] + def which(cmd) + ENV['PATH'] = '' if ENV['PATH'].nil? + paths = (ENV['PATH'].split(::File::PATH_SEPARATOR) + %w(/bin /usr/bin /sbin /usr/sbin)) + + paths.each do |path| + possible = File.join(path, cmd) + return possible if File.executable?(possible) + end + + nil + end + end +end + +Chef::Recipe.send(:include, ::Apt::Helpers) +Chef::Resource.send(:include, ::Apt::Helpers) +Chef::Provider.send(:include, ::Apt::Helpers) diff --git a/berks-cookbooks/apt/metadata.json b/berks-cookbooks/apt/metadata.json new file mode 100644 index 0000000..5d1bda9 --- /dev/null +++ b/berks-cookbooks/apt/metadata.json @@ -0,0 +1 @@ +{"name":"apt","version":"7.2.0","description":"Configures apt and apt caching.","long_description":"# apt Cookbook\n\n[![Build Status](https://img.shields.io/travis/chef-cookbooks/apt.svg)][travis] [![Cookbook Version](https://img.shields.io/cookbook/v/apt.svg)][cookbook]\n\nThis cookbook includes recipes to execute apt-get update to ensure the local APT package cache is up to date. There are recipes for managing the apt-cacher-ng caching proxy and proxy clients. It also includes a custom resource for pinning packages via /etc/apt/preferences.d.\n\n## Requirements\n\n### Platforms\n\n- Ubuntu 12.04+\n- Debian 7+\n\nMay work with or without modification on other Debian derivatives.\n\n### Chef\n\n- Chef 13.3+\n\n### Cookbooks\n\n- None\n\n## Recipes\n\n### default\n\nThis recipe manually updates the timestamp file used to only run `apt-get update` if the cache is more than one day old.\n\nThis recipe should appear first in the run list of Debian or Ubuntu nodes to ensure that the package cache is up to date before managing any `package` resources with Chef.\n\nThis recipe also sets up a local cache directory for preseeding packages.\n\n**Including the default recipe on a node that does not support apt (such as Windows or RHEL) results in a noop.**\n\n### cacher-client\n\nConfigures the node to use a `apt-cacher-ng` server to cache apt requests. Configuration of the server to use is located in `default['apt']['cacher_client']['cacher_server']` which is a hash containing `host`, `port`, `proxy_ssl`, and `bypass` keys. Example:\n\n```json\n{\n \"apt\": {\n \"cacher_client\": {\n \"cacher_server\": {\n \"host\": \"cache_server.mycorp.dmz\",\n \"port\": 1234,\n \"proxy_ssl\": true,\n \"cache_bypass\": {\n \"download.oracle.com\": \"http\"\n }\n }\n }\n }\n}\n```\n\n#### Bypassing the cache\n\nOccasionally you may come across repositories that do not play nicely when the node is using an `apt-cacher-ng` server. You can configure `cacher-client` to bypass the server and connect directly to the repository with the `cache_bypass` attribute.\n\nTo do this, you need to override the `cache_bypass` attribute with an hash of repositories, with each key as the repository URL and value as the protocol to use:\n\n```json\n{\n \"apt\": {\n \"cacher_client\": {\n \"cacher_server\": {\n \"cache_bypass\": {\n \"URL\": \"PROTOCOL\"\n }\n }\n }\n }\n}\n```\n\nFor example, to prevent caching and directly connect to the repository at `download.oracle.com` via http and the repo at `nginx.org` via https\n\n```json\n{\n \"apt\": {\n \"cacher_client\": {\n \"cacher_server\": {\n \"cache_bypass\": {\n \"download.oracle.com\": \"http\",\n \"nginx.org\": \"https\"\n }\n }\n }\n }\n}\n```\n\n### cacher-ng\n\nInstalls the `apt-cacher-ng` package and service so the system can provide APT caching. You can check the usage report at .\n\nIf you wish to help the `cacher-ng` recipe seed itself, you must now explicitly include the `cacher-client` recipe in your run list **after** `cacher-ng` or you will block your ability to install any packages (ie. `apt-cacher-ng`).\n\n### unattended-upgrades\n\nInstalls and configures the `unattended-upgrades` package to provide automatic package updates. This can be configured to upgrade all packages or to just install security updates by setting `['apt']['unattended_upgrades']['allowed_origins']`.\n\nTo pull just security updates, set `origins_patterns` to something like `[\"origin=Ubuntu,archive=trusty-security\"]` (for Ubuntu trusty) or `[\"origin=Debian,label=Debian-Security\"]` (for Debian).\n\n## Attributes\n\n### General\n\n- `['apt']['compile_time_update']` - force the default recipe to run `apt-get update` at compile time.\n- `['apt']['periodic_update_min_delay']` - minimum delay (in seconds) between two actual executions of `apt-get update` by the `execute[apt-get-update-periodic]` resource, default is '86400' (24 hours)\n\n### Caching\n\n- `['apt']['cacher_client']['cacher_server']` - Hash containing server information used by clients for caching. See the example in the recipes section above for the full format of the hash.\n- `['apt']['cacher_interface']` - interface to connect to the cacher-ng service, no default.\n- `['apt']['cacher_port']` - port for the cacher-ng service (used by server recipe only), default is '3142'\n- `['apt']['cacher_dir']` - directory used by cacher-ng service, default is '/var/cache/apt-cacher-ng'\n- `['apt']['compiletime']` - force the `cacher-client` recipe to run before other recipes. It forces apt to use the proxy before other recipes run. Useful if your nodes have limited access to public apt repositories. This is overridden if the `cacher-ng` recipe is in your run list. Default is 'false'\n\n### Unattended Upgrades\n\n- `['apt']['unattended_upgrades']['enable']` - enables unattended upgrades, default is false\n- `['apt']['unattended_upgrades']['update_package_lists']` - automatically update package list (`apt-get update`) daily, default is true\n- `['apt']['unattended_upgrades']['allowed_origins']` - array of allowed apt origins from which to pull automatic upgrades, defaults to a guess at the system's main origin and should almost always be overridden\n- `['apt']['unattended_upgrades']['origins_patterns']` - array of allowed apt origin patterns from which to pull automatic upgrades, defaults to none.\n- `['apt']['unattended_upgrades']['package_blacklist']` - an array of package which should never be automatically upgraded, defaults to none\n- `['apt']['unattended_upgrades']['auto_fix_interrupted_dpkg']` - attempts to repair dpkg state with `dpkg --force-confold --configure -a` if it exits uncleanly, defaults to false (contrary to the unattended-upgrades default)\n- `['apt']['unattended_upgrades']['minimal_steps']` - Split the upgrade into the smallest possible chunks. This makes the upgrade a bit slower but it has the benefit that shutdown while a upgrade is running is possible (with a small delay). Defaults to false.\n- `['apt']['unattended_upgrades']['install_on_shutdown']` - Install upgrades when the machine is shuting down instead of doing it in the background while the machine is running. This will (obviously) make shutdown slower. Defaults to false.\n- `['apt']['unattended_upgrades']['mail']` - Send email to this address for problems or packages upgrades. Defaults to no email.\n- `['apt']['unattended_upgrades']['sender']` - Send email from this address for problems or packages upgrades. Defaults to 'root'.\n- `['apt']['unattended_upgrades']['mail_only_on_error']` - If set, email will only be set on upgrade errors. Otherwise, an email will be sent after each upgrade. Defaults to true.\n- `['apt']['unattended_upgrades']['remove_unused_dependencies']` Do automatic removal of new unused dependencies after the upgrade. Defaults to false.\n- `['apt']['unattended_upgrades']['automatic_reboot']` - Automatically reboots _without confirmation_ if a restart is required after the upgrade. Defaults to false.\n- `['apt']['unattended_upgrades']['dl_limit']` - Limits the bandwidth used by apt to download packages. Value given as an integer in kb/sec. Defaults to nil (no limit).\n- `['apt']['unattended_upgrades']['random_sleep']` - Wait a random number of seconds up to this value before running daily periodic apt actions. System default is 1800 seconds (30 minutes).\n- `['apt']['unattended_upgrades']['syslog_enable']` - Enable logging to syslog. Defaults to false.\n- `['apt']['unattended_upgrades']['syslog_facility']` - Specify syslog facility. Defaults to 'daemon'.\n- `['apt']['unattended_upgrades']['dpkg_options']` An array of dpkg options to be used specifically only for unattended upgrades. Defaults to `[]` which will prevent it from being rendered from the template in the resulting file.\n\n### Configuration for APT\n\n- `['apt']['confd']['force_confask']` - Prompt when overwriting configuration files. (default: false)\n- `['apt']['confd']['force_confdef']` - Don't prompt when overwriting configuration files. (default: false)\n- `['apt']['confd']['force_confmiss']` - Install removed configuration files when upgrading packages. (default: false)\n- `['apt']['confd']['force_confnew']` - Overwrite configuration files when installing packages. (default: false)\n- `['apt']['confd']['force_confold']` - Keep modified configuration files when installing packages. (default: false)\n- `['apt']['confd']['install_recommends']` - Consider recommended packages as a dependency for installing. (default: true)\n- `['apt']['confd']['install_suggests']` - Consider suggested packages as a dependency for installing. (default: false)\n\n## Libraries\n\nThere is an `interface_ipaddress` method that returns the IP address for a particular host and interface, used by the `cacher-client` recipe. To enable it on the server use the `['apt']['cacher_interface']` attribute.\n\n## Usage\n\nPut `recipe[apt]` first in the run list. If you have other recipes that you want to use to configure how apt behaves, like new sources, notify the execute resource to run, e.g.:\n\n```ruby\ntemplate '/etc/apt/sources.list.d/my_apt_sources.list' do\n notifies :run, 'execute[apt-get update]', :immediately\nend\n```\n\nThe above will run during execution phase since it is a normal template resource, and should appear before other package resources that need the sources in the template.\n\nPut `recipe[apt::cacher-ng]` in the run_list for a server to provide APT caching and add `recipe[apt::cacher-client]` on the rest of the Debian-based nodes to take advantage of the caching server.\n\nIf you want to cleanup unused packages, there is also the `apt-get autoclean` and `apt-get autoremove` resources provided for automated cleanup.\n\n## Resources\n\n### apt_preference\n\nThe apt_preference resource has been moved into chef-client in Chef 13.3.\n\nSee for usage details\n\n### apt_repository\n\nThe apt_repository resource has been moved into chef-client in Chef 12.9.\n\nSee for usage details\n\n### apt_update\n\nThe apt_update resource has been moved into chef-client in Chef 12.7.\n\nSee for usage details\n\n## Maintainers\n\nThis cookbook is maintained by Chef's Community Cookbook Engineering team. Our goal is to improve cookbook quality and to aid the community in contributing to cookbooks. To learn more about our team, process, and design goals see our [team documentation](https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/COOKBOOK_TEAM.MD). To learn more about contributing to cookbooks like this see our [contributing documentation](https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD), or if you have general questions about this cookbook come chat with us in #cookbok-engineering on the [Chef Community Slack](http://community-slack.chef.io/)\n\n## License\n\n**Copyright:** 2009-2017, Chef Software, Inc.\n\n```\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n\n[cookbook]: https://community.chef.io/cookbooks/apt\n[travis]: https://travis-ci.org/chef-cookbooks/apt\n","maintainer":"Chef Software, Inc.","maintainer_email":"cookbooks@chef.io","license":"Apache-2.0","platforms":{"ubuntu":">= 0.0.0","debian":">= 0.0.0"},"dependencies":{},"recommendations":{},"suggestions":{},"conflicting":{},"providing":{},"replacing":{},"attributes":{},"groupings":{},"recipes":{},"source_url":"https://github.com/chef-cookbooks/apt","issues_url":"https://github.com/chef-cookbooks/apt/issues","chef_version":[[">= 13.3"]],"ohai_version":[]} \ No newline at end of file diff --git a/berks-cookbooks/apt/metadata.rb b/berks-cookbooks/apt/metadata.rb new file mode 100644 index 0000000..05fbd21 --- /dev/null +++ b/berks-cookbooks/apt/metadata.rb @@ -0,0 +1,15 @@ +name 'apt' +maintainer 'Chef Software, Inc.' +maintainer_email 'cookbooks@chef.io' +license 'Apache-2.0' +description 'Configures apt and apt caching.' +long_description IO.read(File.join(File.dirname(__FILE__), 'README.md')) +version '7.2.0' + +%w(ubuntu debian).each do |os| + supports os +end + +source_url 'https://github.com/chef-cookbooks/apt' +issues_url 'https://github.com/chef-cookbooks/apt/issues' +chef_version '>= 13.3' diff --git a/berks-cookbooks/apt/recipes/cacher-client.rb b/berks-cookbooks/apt/recipes/cacher-client.rb new file mode 100644 index 0000000..1bbf92c --- /dev/null +++ b/berks-cookbooks/apt/recipes/cacher-client.rb @@ -0,0 +1,52 @@ +# +# Cookbook:: apt +# Recipe:: cacher-client +# +# Copyright:: 2011-2017, Chef Software, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# remove Acquire::http::Proxy lines from /etc/apt/apt.conf since we use 01proxy +# these are leftover from preseed installs +execute 'Remove proxy from /etc/apt/apt.conf' do + command "sed --in-place '/^Acquire::http::Proxy/d' /etc/apt/apt.conf" + only_if 'grep Acquire::http::Proxy /etc/apt/apt.conf' +end + +if node['apt']['cacher_client']['cacher_server'].empty? + Chef::Log.warn("No cache server defined in node['apt']['cacher_client']['cacher_server']. Not setting up caching") + f = file '/etc/apt/apt.conf.d/01proxy' do + action(node['apt']['compiletime'] ? :nothing : :delete) + end + f.run_action(:delete) if node['apt']['compiletime'] +else + apt_update 'update for notification' do + action :nothing + end + + t = template '/etc/apt/apt.conf.d/01proxy' do + source '01proxy.erb' + owner 'root' + group 'root' + mode '0644' + variables( + server: node['apt']['cacher_client']['cacher_server'] + ) + action(node['apt']['compiletime'] ? :nothing : :create) + notifies :update, 'apt_update[update for notification]', :immediately + end + t.run_action(:create) if node['apt']['compiletime'] +end + +include_recipe 'apt::default' diff --git a/berks-cookbooks/apt/recipes/cacher-ng.rb b/berks-cookbooks/apt/recipes/cacher-ng.rb new file mode 100644 index 0000000..5d7e846 --- /dev/null +++ b/berks-cookbooks/apt/recipes/cacher-ng.rb @@ -0,0 +1,39 @@ +# +# Cookbook:: apt +# Recipe:: cacher-ng +# +# Copyright:: 2008-2017, Chef Software, Inc. +# +# Licensed under the Apache License, Version 2.0 (the 'License'); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an 'AS IS' BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +package 'apt-cacher-ng' + +directory node['apt']['cacher_dir'] do + owner 'apt-cacher-ng' + group 'apt-cacher-ng' + mode '0755' +end + +template '/etc/apt-cacher-ng/acng.conf' do + source 'acng.conf.erb' + owner 'root' + group 'root' + mode '0644' + notifies :restart, 'service[apt-cacher-ng]', :immediately +end + +service 'apt-cacher-ng' do + supports restart: true, status: false + action [:enable, :start] +end diff --git a/berks-cookbooks/apt/recipes/default.rb b/berks-cookbooks/apt/recipes/default.rb new file mode 100644 index 0000000..e8c9e8c --- /dev/null +++ b/berks-cookbooks/apt/recipes/default.rb @@ -0,0 +1,98 @@ +# +# Cookbook:: apt +# Recipe:: default +# +# Copyright:: 2008-2017, Chef Software, Inc. +# Copyright:: 2009-2017, Bryan McLellan +# +# Licensed under the Apache License, Version 2.0 (the 'License'); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an 'AS IS' BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# On systems where apt is not installed, the resources in this recipe are not +# executed. However, they _must_ still be present in the resource collection +# or other cookbooks which notify these resources will fail on non-apt-enabled +# systems. + +file '/var/lib/apt/periodic/update-success-stamp' do + owner 'root' + group 'root' + action :nothing +end + +# If compile_time_update run apt-get update at compile time +if node['apt']['compile_time_update'] && apt_installed? + apt_update('compile time') do + frequency node['apt']['periodic_update_min_delay'] + ignore_failure true + end.run_action(:periodic) +end + +apt_update 'periodic' do + frequency node['apt']['periodic_update_min_delay'] +end + +# For other recipes to call to force an update +execute 'apt-get update' do + command 'apt-get update' + ignore_failure true + action :nothing + notifies :touch, 'file[/var/lib/apt/periodic/update-success-stamp]', :immediately + only_if { apt_installed? } +end + +# Automatically remove packages that are no longer needed for dependencies +execute 'apt-get autoremove' do + command 'apt-get -y autoremove' + environment( + 'DEBIAN_FRONTEND' => 'noninteractive' + ) + action :nothing + only_if { apt_installed? } +end + +# Automatically remove .deb files for packages no longer on your system +execute 'apt-get autoclean' do + command 'apt-get -y autoclean' + action :nothing + only_if { apt_installed? } +end + +%w(/var/cache/local /var/cache/local/preseeding).each do |dirname| + directory dirname do + owner 'root' + group 'root' + mode '0755' + action :create + only_if { apt_installed? } + end +end + +template '/etc/apt/apt.conf.d/10dpkg-options' do + owner 'root' + group 'root' + mode '0644' + source '10dpkg-options.erb' + only_if { apt_installed? } +end + +template '/etc/apt/apt.conf.d/10recommends' do + owner 'root' + group 'root' + mode '0644' + source '10recommends.erb' + only_if { apt_installed? } +end + +package %w(apt-transport-https gnupg dirmngr) do + only_if { apt_installed? } +end diff --git a/berks-cookbooks/apt/recipes/unattended-upgrades.rb b/berks-cookbooks/apt/recipes/unattended-upgrades.rb new file mode 100644 index 0000000..05c3b89 --- /dev/null +++ b/berks-cookbooks/apt/recipes/unattended-upgrades.rb @@ -0,0 +1,47 @@ +# +# Cookbook:: apt +# Recipe:: unattended-upgrades +# +# Copyright:: 2014-2017, Chef Software, Inc. +# +# Licensed under the Apache License, Version 2.0 (the 'License'); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an 'AS IS' BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# On systems where apt is not installed, the resources in this recipe are not +# executed. However, they _must_ still be present in the resource collection +# or other cookbooks which notify these resources will fail on non-apt-enabled +# systems. +# + +package 'unattended-upgrades' do # ~FC009 + response_file 'unattended-upgrades.seed.erb' + action :install +end + +package 'bsd-mailx' do + not_if { node['apt']['unattended_upgrades']['mail'].nil? } +end + +template '/etc/apt/apt.conf.d/20auto-upgrades' do + owner 'root' + group 'root' + mode '0644' + source '20auto-upgrades.erb' +end + +template '/etc/apt/apt.conf.d/50unattended-upgrades' do + owner 'root' + group 'root' + mode '0644' + source '50unattended-upgrades.erb' +end diff --git a/berks-cookbooks/apt/templates/01proxy.erb b/berks-cookbooks/apt/templates/01proxy.erb new file mode 100644 index 0000000..1ede9b3 --- /dev/null +++ b/berks-cookbooks/apt/templates/01proxy.erb @@ -0,0 +1,11 @@ +Acquire::http::Proxy "http://<%= @server['host'] %>:<%= @server['port'] %>"; +<% if @server['proxy_ssl'] %> +Acquire::https::Proxy "http://<%= @server['host'] %>:<%= @server['port'] %>"; +<% else %> +Acquire::https::Proxy "DIRECT"; +<% end %> +<% unless @server['cache_bypass'].nil? %> +<% @server['cache_bypass'].each do |bypass, type| %> +Acquire::<%= type %>::Proxy::<%= bypass %> "DIRECT"; +<% end %> +<% end %> diff --git a/berks-cookbooks/apt/templates/10dpkg-options.erb b/berks-cookbooks/apt/templates/10dpkg-options.erb new file mode 100644 index 0000000..3111026 --- /dev/null +++ b/berks-cookbooks/apt/templates/10dpkg-options.erb @@ -0,0 +1,8 @@ +# Managed by Chef +DPkg::Options { +<%= node['apt']['confd']['force_confask'] ? '"--force-confask";' : '' -%> +<%= node['apt']['confd']['force_confdef'] ? '"--force-confdef";' : '' -%> +<%= node['apt']['confd']['force_confmiss'] ? '"--force-confmiss";' : '' -%> +<%= node['apt']['confd']['force_confnew'] ? '"--force-confnew";' : '' -%> +<%= node['apt']['confd']['force_confold'] ? '"--force-confold";' : '' -%> +} diff --git a/berks-cookbooks/apt/templates/10recommends.erb b/berks-cookbooks/apt/templates/10recommends.erb new file mode 100644 index 0000000..16b3664 --- /dev/null +++ b/berks-cookbooks/apt/templates/10recommends.erb @@ -0,0 +1,3 @@ +# Managed by Chef +APT::Install-Recommends "<%= node['apt']['confd']['install_recommends'] ? 1 : 0 %>"; +APT::Install-Suggests "<%= node['apt']['confd']['install_suggests'] ? 1 : 0 %>"; diff --git a/berks-cookbooks/apt/templates/20auto-upgrades.erb b/berks-cookbooks/apt/templates/20auto-upgrades.erb new file mode 100644 index 0000000..47b2f23 --- /dev/null +++ b/berks-cookbooks/apt/templates/20auto-upgrades.erb @@ -0,0 +1,5 @@ +APT::Periodic::Update-Package-Lists "<%= node['apt']['unattended_upgrades']['update_package_lists'] ? 1 : 0 %>"; +APT::Periodic::Unattended-Upgrade "<%= node['apt']['unattended_upgrades']['enable'] ? 1 : 0 %>"; +<% if node['apt']['unattended_upgrades']['random_sleep'] -%> +APT::Periodic::RandomSleep "<%= node['apt']['unattended_upgrades']['random_sleep'] %>"; +<% end -%> diff --git a/berks-cookbooks/apt/templates/50unattended-upgrades.erb b/berks-cookbooks/apt/templates/50unattended-upgrades.erb new file mode 100644 index 0000000..de9cd82 --- /dev/null +++ b/berks-cookbooks/apt/templates/50unattended-upgrades.erb @@ -0,0 +1,104 @@ +// Automatically upgrade packages from these (origin:archive) pairs +Unattended-Upgrade::Allowed-Origins { +<% unless node['apt']['unattended_upgrades']['allowed_origins'].empty? -%> +<% node['apt']['unattended_upgrades']['allowed_origins'].each do |origin| -%> + "<%= origin %>"; +<% end -%> +<% end -%> +}; + +<% unless node['apt']['unattended_upgrades']['origins_patterns'].empty? -%> +Unattended-Upgrade::Origins-Pattern { +<% node['apt']['unattended_upgrades']['origins_patterns'].each do |pattern| -%> + "<%= pattern %>"; +<% end -%> +}; + +<% end -%> +// List of packages to not update +Unattended-Upgrade::Package-Blacklist { +<% unless node['apt']['unattended_upgrades']['package_blacklist'].empty? -%> +<% node['apt']['unattended_upgrades']['package_blacklist'].each do |package| -%> + "<%= package %>"; +<% end -%> +<% end -%> +}; + +// This option allows you to control if on a unclean dpkg exit +// unattended-upgrades will automatically run +// dpkg --force-confold --configure -a +// The default is true, to ensure updates keep getting installed +Unattended-Upgrade::AutoFixInterruptedDpkg "<%= node['apt']['unattended_upgrades']['auto_fix_interrupted_dpkg'] ? 'true' : 'false' %>"; + +// Split the upgrade into the smallest possible chunks so that +// they can be interrupted with SIGUSR1. This makes the upgrade +// a bit slower but it has the benefit that shutdown while a upgrade +// is running is possible (with a small delay) +Unattended-Upgrade::MinimalSteps "<%= node['apt']['unattended_upgrades']['minimal_steps'] ? 'true' : 'false' %>"; + +// Install all unattended-upgrades when the machine is shuting down +// instead of doing it in the background while the machine is running +// This will (obviously) make shutdown slower +Unattended-Upgrade::InstallOnShutdown "<%= node['apt']['unattended_upgrades']['install_on_shutdown'] ? 'true' : 'false' %>"; + +<% if node['apt']['unattended_upgrades']['mail'] -%> +// Send email to this address for problems or packages upgrades +// If empty or unset then no email is sent, make sure that you +// have a working mail setup on your system. A package that provides +// 'mailx' must be installed. +Unattended-Upgrade::Mail "<%= node['apt']['unattended_upgrades']['mail'] %>"; +<% end -%> + +<% if node['apt']['unattended_upgrades']['sender'] -%> +// This option allows to customize the email address used in the +// 'From' header. unattended-upgrades will use "root" if unset. +Unattended-Upgrade::Sender "<%= node['apt']['unattended_upgrades']['sender'] %>"; +<% end -%> + +// Set this value to "true" to get emails only on errors. Default +// is to always send a mail if Unattended-Upgrade::Mail is set +Unattended-Upgrade::MailOnlyOnError "<%= node['apt']['unattended_upgrades']['mail_only_on_error'] ? 'true' : 'false' %>"; + +// Do automatic removal of new unused dependencies after the upgrade +// (equivalent to apt-get autoremove) +Unattended-Upgrade::Remove-Unused-Dependencies "<%= node['apt']['unattended_upgrades']['remove_unused_dependencies'] ? 'true' : 'false' %>"; + +// Automatically reboot *WITHOUT CONFIRMATION* if a +// the file /var/run/reboot-required is found after the upgrade +Unattended-Upgrade::Automatic-Reboot "<%= node['apt']['unattended_upgrades']['automatic_reboot'] ? 'true' : 'false' %>"; + +<% if node['apt']['unattended_upgrades']['automatic_reboot'] -%> +// If automatic reboot is enabled and needed, reboot at the specific +// time instead of immediately. Default is "now" +Unattended-Upgrade::Automatic-Reboot-Time "<%= node['apt']['unattended_upgrades']['automatic_reboot_time'] %>"; +<% end %> + +<% if node['apt']['unattended_upgrades']['dl_limit'] -%> +// Use apt bandwidth limit feature, this example limits the download +// speed to 70kb/sec +// Acquire::http::Dl-Limit "70"; +Acquire::http::Dl-Limit "<%= node['apt']['unattended_upgrades']['dl_limit'] %>"; +<% end -%> + +// Enable logging to syslog. Default is False +Unattended-Upgrade::SyslogEnable "<%= node['apt']['unattended_upgrades']['syslog_enable'] ? 'true' : 'false' %>"; + +// Specify syslog facility. Default is daemon +Unattended-Upgrade::SyslogFacility "<%= node['apt']['unattended_upgrades']['syslog_facility'] %>"; + +// specify any dpkg options you want to run +// for example if you wanted to upgrade and use +// the installed version of config files when +// resolving conflicts during an upgrade you +// typically need: +// Dpkg::Options { +// "--force-confdef"; +// "--force-confold"; +//}; +<% unless node['apt']['unattended_upgrades']['dpkg_options'].empty? -%> +Dpkg::Options { +<% node['apt']['unattended_upgrades']['dpkg_options'].each do |option|%> + "<%= option %>"; +<% end -%> +}; +<% end -%> diff --git a/berks-cookbooks/apt/templates/acng.conf.erb b/berks-cookbooks/apt/templates/acng.conf.erb new file mode 100644 index 0000000..3aa0c92 --- /dev/null +++ b/berks-cookbooks/apt/templates/acng.conf.erb @@ -0,0 +1,275 @@ +# Letter case in directive names does not matter. Must be separated with colons. +# Valid boolean values are a zero number for false, non-zero numbers for true. + +CacheDir: <%= node['apt']['cacher_dir'] %> + +# set empty to disable logging +LogDir: /var/log/apt-cacher-ng + +# place to look for additional configuration and resource files if they are not +# found in the configuration directory +# SupportDir: /usr/lib/apt-cacher-ng + +# TCP (http) port +# Set to 9999 to emulate apt-proxy +Port:<%= node['apt']['cacher_port'] %> + +# Addresses or hostnames to listen on. Multiple addresses must be separated by +# spaces. Each entry must be an exact local address which is associated with a +# local interface. DNS resolution is performed using getaddrinfo(3) for all +# available protocols (IPv4, IPv6, ...). Using a protocol specific format will +# create binding(s) only on protocol specific socket(s) (e.g. 0.0.0.0 will listen +# only to IPv4). +# +# Default: not set, will listen on all interfaces and protocols +# +# BindAddress: localhost 192.168.7.254 publicNameOnMainInterface + +# The specification of another proxy which shall be used for downloads. +# Username and password are, and see manual for limitations. +# +#Proxy: http://www-proxy.example.net:80 +#proxy: username:proxypassword@proxy.example.net:3128 + +# Repository remapping. See manual for details. +# In this example, some backends files might be generated during package +# installation using information collected on the system. +Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian # Debian Archives +Remap-uburep: file:ubuntu_mirrors /ubuntu ; file:backends_ubuntu # Ubuntu Archives +Remap-debvol: file:debvol_mirror*.gz /debian-volatile ; file:backends_debvol # Debian Volatile Archives +Remap-cygwin: file:cygwin_mirrors /cygwin # ; file:backends_cygwin # incomplete, please create this file or specify preferred mirrors here +Remap-sfnet: file:sfnet_mirrors # ; file:backends_sfnet # incomplete, please create this file or specify preferred mirrors here +Remap-alxrep: file:archlx_mirrors /archlinux # ; file:backend_archlx # Arch Linux +Remap-fedora: file:fedora_mirrors # Fedora Linux +Remap-epel: file:epel_mirrors # Fedora EPEL +Remap-slrep: file:sl_mirrors # Scientific Linux + +# This is usually not needed for security.debian.org because it's always the +# same DNS hostname. However, it might be enabled in order to use hooks, +# ForceManaged mode or special flags in this context. +# Remap-secdeb: security.debian.org + +# Virtual page accessible in a web browser to see statistics and status +# information, i.e. under http://localhost:3142/acng-report.html +ReportPage: acng-report.html + +# Socket file for accessing through local UNIX socket instead of TCP/IP. Can be +# used with inetd bridge or cron client. +# SocketPath:/var/run/apt-cacher-ng/socket + +# Forces log file to be written to disk after every line when set to 1. Default +# is 0, buffers are flushed when the client disconnects. +# +# (technically, alias to the Debug option, see its documentation for details) +# +# UnbufferLogs: 0 + +# Set to 0 to store only type, time and transfer sizes. +# 1 -> client IP and relative local path are logged too +# VerboseLog: 1 + +# Don't detach from the console +# ForeGround: 0 + +# Store the pid of the daemon process therein +# PidFile: /var/run/apt-cacher-ng/pid + +# Forbid outgoing connections, work around them or respond with 503 error +# offlinemode:0 + +# Forbid all downloads that don't run through preconfigured backends (.where) +#ForceManaged: 0 + +# Days before considering an unreferenced file expired (to be deleted). +# Warning: if the value is set too low and particular index files are not +# available for some days (mirror downtime) there is a risk of deletion of +# still useful package files. +ExTreshold: 4 + +# Stop expiration when a critical problem appeared. Currently only failed +# refresh of an index file is considered as critical. +# +# WARNING: don't touch this option or set to zero. +# Anything else is DANGEROUS and may cause data loss. +# +# ExAbortOnProblems: 1 + +# Replace some Windows/DOS-FS incompatible chars when storing +# StupidFs: 0 + +# Experimental feature for apt-listbugs: pass-through SOAP requests and +# responses to/from bugs.debian.org. If not set, default is true if +# ForceManaged is enabled and false otherwise. +# ForwardBtsSoap: 1 + +# The daemon has a small cache for DNS data, to speed up resolution. The +# expiration time of the DNS entries can be configured in seconds. +# DnsCacheSeconds: 3600 + +# Don't touch the following values without good consideration! +# +# Max. count of connection threads kept ready (for faster response in the +# future). Should be a sane value between 0 and average number of connections, +# and depend on the amount of spare RAM. +# MaxStandbyConThreads: 8 +# +# Hard limit of active thread count for incoming connections, i.e. operation +# is refused when this value is reached (below zero = unlimited). +# MaxConThreads: -1 +# +# Pigeonholing files with regular expressions (static/volatile). Can be +# overriden here but not should not be done permanently because future update +# of default settings would not be applied later. +# VfilePattern = (^|.*?/)(Index|Packages(\.gz|\.bz2|\.lzma|\.xz)?|InRelease|Release|Release\.gpg|Sources(\.gz|\.bz2|\.lzma|\.xz)?|release|index\.db-.*\.gz|Contents-[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|/meta-release[^/]*|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|((setup|setup-legacy)(\.ini|\.bz2|\.hint)(\.sig)?)|mirrors\.lst|repo(index|md)\.xml(\.asc|\.key)?|directory\.yast|products|content(\.asc|\.key)?|media|filelists\.xml\.gz|filelists\.sqlite\.bz2|repomd\.xml|packages\.[a-zA-Z][a-zA-Z]\.gz|info\.txt|license\.tar\.gz|license\.zip|.*\.db(\.tar\.gz)?|.*\.files\.tar\.gz|.*\.abs\.tar\.gz|metalink\?repo|.*prestodelta\.xml\.gz)$|/dists/.*/installer-[^/]+/[^0-9][^/]+/images/.* +# PfilePattern = .*(\.d?deb|\.rpm|\.dsc|\.tar(\.gz|\.bz2|\.lzma|\.xz)(\.gpg)?|\.diff(\.gz|\.bz2|\.lzma|\.xz)|\.jigdo|\.template|changelog|copyright|\.udeb|\.debdelta|\.diff/.*\.gz|(Devel)?ReleaseAnnouncement(\?.*)?|[a-f0-9]+-(susedata|updateinfo|primary|deltainfo).xml.gz|fonts/(final/)?[a-z]+32.exe(\?download.*)?|/dists/.*/installer-[^/]+/[0-9][^/]+/images/.*)$ +# Whitelist for expiration, file types not to be removed even when being +# unreferenced. Default: many parts from VfilePattern where no parent index +# exists or might be unknown. +# WfilePattern = (^|.*?/)(Release|InRelease|Release\.gpg|(Packages|Sources)(\.gz|\.bz2|\.lzma|\.xz)?|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|.*\.xml|.*\.db\.tar\.gz|.*\.files\.tar\.gz|.*\.abs\.tar\.gz|[a-z]+32.exe)$|/dists/.*/installer-.*/images/.* + +# Higher modes only working with the debug version +# Warning, writes a lot into apt-cacher.err logfile +# Value overwrites UnbufferLogs setting (aliased) +# Debug:3 + +# Usually, general purpose proxies like Squid expose the IP address of the +# client user to the remote server using the X-Forwarded-For HTTP header. This +# behaviour can be optionally turned on with the Expose-Origin option. +# ExposeOrigin: 0 + +# When logging the originating IP address, trust the information supplied by +# the client in the X-Forwarded-For header. +# LogSubmittedOrigin: 0 + +# The version string reported to the peer, to be displayed as HTTP client (and +# version) in the logs of the mirror. +# WARNING: some archives use this header to detect/guess capabilities of the +# client (i.e. redirection support) and change the behaviour accordingly, while +# ACNG might not support the expected features. Expect side effects. +# +# UserAgent: Yet Another HTTP Client/1.2.3p4 + +# In some cases the Import and Expiration tasks might create fresh volatile +# data for internal use by reconstructing them using patch files. This +# by-product might be recompressed with bzip2 and with some luck the resulting +# file becomes identical to the *.bz2 file on the server, usable for APT +# clients trying to fetch the full .bz2 compressed version. Injection of the +# generated files into the cache has however a disadvantage on underpowered +# servers: bzip2 compression can create high load on the server system and the +# visible download of the busy .bz2 files also becomes slower. +# +# RecompBz2: 0 + +# Network timeout for outgoing connections. +# NetworkTimeout: 60 + +# Sometimes it makes sense to not store the data in cache and just return the +# package data to client as it comes in. DontCache parameters can enable this +# behaviour for certain URL types. The tokens are extended regular expressions +# that URLs are matched against. +# +# DontCacheRequested is applied to the URL as it comes in from the client. +# Example: exclude packages built with kernel-package for x86 +# DontCacheRequested: linux-.*_10\...\.Custo._i386 +# Example usecase: exclude popular private IP ranges from caching +# DontCacheRequested: 192.168.0 ^10\..* 172.30 +# +# DontCacheResolved is applied to URLs after mapping to the target server. If +# multiple backend servers are specified then it's only matched against the +# download link for the FIRST possible source (due to implementation limits). +# Example usecase: all Ubuntu stuff comes from a local mirror (specified as +# backend), don't cache it again: +# DontCacheResolved: ubuntumirror.local.net +# +# DontCache directive sets (overrides) both, DontCacheResolved and +# DontCacheRequested. Provided for convenience, see those directives for +# details. +# +# Default permission set of freshly created files and directories, as octal +# numbers (see chmod(1) for details). +# Can by limited by the umask value (see umask(2) for details) if it's set in +# the environment of the starting shell, e.g. in apt-cacher-ng init script or +# in its configuration file. +# DirPerms: 00755 +# FilePerms: 00664 +# +# +# It's possible to use use apt-cacher-ng as a regular web server with limited +# feature set, i.e. +# including directory browsing and download of any file; +# excluding sorting, mime types/encodings, CGI execution, index page +# redirection and other funny things. +# To get this behavior, mappings between virtual directories and real +# directories on the server must be defined with the LocalDirs directive. +# Virtual and real dirs are separated by spaces, multiple pairs are separated +# by semi-colons. Real directories must be absolute paths. +# NOTE: Since the names of that key directories share the same namespace as +# repository names (see Remap-...) it's administrators job to avoid such +# collisions on them (unless created deliberately). +# +# LocalDirs: woo /data/debarchive/woody ; hamm /data/debarchive/hamm + +# Precache a set of files referenced by specified index files. This can be used +# to create a partial mirror usable for offline work. There are certain limits +# and restrictions on the path specification, see manual for details. A list of +# (maybe) relevant index files could be retrieved via +# "apt-get --print-uris update" on a client machine. +# +# PrecacheFor: debrep/dists/unstable/*/source/Sources* debrep/dists/unstable/*/binary-amd64/Packages* + +# Arbitrary set of data to append to request headers sent over the wire. Should +# be a well formated HTTP headers part including newlines (DOS style) which +# can be entered as escape sequences (\r\n). +# RequestAppendix: X-Tracking-Choice: do-not-track\r\n + +# Specifies the IP protocol families to use for remote connections. Order does +# matter, first specified are considered first. Possible combinations: +# v6 v4 +# v4 v6 +# v6 +# v4 +# (empty or not set: use system default) +# +# ConnectProto: v6 v4 + +# Regular expiration algorithm finds package files which are no longer listed +# in any index file and removes them of them after a safety period. +# This option allows to keep more versions of a package in the cache after +# safety period is over. +# KeepExtraVersions: 1 + +# Optionally uses TCP access control provided by libwrap, see hosts_access(5) +# for details. Daemon name is apt-cacher-ng. Default if not set: decided on +# startup by looking for explicit mentioning of apt-cacher-ng in +# /etc/hosts.allow or /etc/hosts.deny files. +# UseWrap: 0 + +# If many machines from the same local network attempt to update index files +# (apt-get update) at nearly the same time, the known state of these index file +# is temporarily frozen and multiple requests receive the cached response +# without contacting the server. This parameter (in seconds) specifies the +# length of this period before the files are considered outdated. +# Setting it too low transfers more data and increases remote server load, +# setting it too high (more than a couple of minutes) increases the risk of +# delivering inconsistent responses to the clients. +# FreshIndexMaxAge: 27 + +# Usually the users are not allowed to specify custom TCP ports of remote +# mirrors in the requests, only the default HTTP port can be used (instead, +# proxy administrator can create Remap- rules with custom ports). This +# restriction can be disabled by specifying a list of allowed ports or 0 for +# any port. +# +# AllowUserPorts: 80 + +# Normally the HTTP redirection responses are forwarded to the original caller +# (i.e. APT) which starts a new download attempt from the new URL. This +# solution is ok for client configurations with proxy mode but doesn't work +# well with configurations using URL prefixes. To work around this the server +# can restart its own download with another URL. However, this might be used to +# circumvent download source policies by malicious users. +# The RedirMax option specifies how many such redirects the server should +# follow per request, 0 disables the internal redirection. If not set, +# default value is 0 if ForceManaged is used and 5 otherwise. +# +# RedirMax: 5 diff --git a/berks-cookbooks/apt/templates/unattended-upgrades.seed.erb b/berks-cookbooks/apt/templates/unattended-upgrades.seed.erb new file mode 100644 index 0000000..5ee5e93 --- /dev/null +++ b/berks-cookbooks/apt/templates/unattended-upgrades.seed.erb @@ -0,0 +1 @@ +unattended-upgrades unattended-upgrades/enable_auto_updates boolean <%= node['apt']['unattended_upgrades']['enable'] ? 'true' : 'false' %> diff --git a/chef_runlist b/chef_runlist new file mode 100644 index 0000000..800e6b5 --- /dev/null +++ b/chef_runlist @@ -0,0 +1,6 @@ +"recipe[apt::default]", +"recipe[midterm-vuln::system_config]", +"recipe[midterm-vuln::openvpn]", +"recipe[midterm-vuln::sshd]", +"recipe[midterm-vuln::users]", +"recipe[midterm-vuln::clear_cache]" \ No newline at end of file diff --git a/cookbooks/midterm-vuln/attributes/default.rb b/cookbooks/midterm-vuln/attributes/default.rb new file mode 100644 index 0000000..dc4dbf4 --- /dev/null +++ b/cookbooks/midterm-vuln/attributes/default.rb @@ -0,0 +1,18 @@ +# +# Cookbook:: metasploitable +# Attributes:: default +# + +default[:metasploitable][:files_path] = '/vagrant/chef/cookbooks/metasploitable/files/' + +default[:metasploitable][:ports] = { :cups => 631, + :apache => 80, + :unrealircd => 6697, + :proftpd => 21, + :mysql => 3306, + :chatbot_ui => 80, + :chatbot_nodejs => 3000, + :readme_app => 3500, + :sinatra => 8181, + :samba => 445 +} diff --git a/cookbooks/midterm-vuln/attributes/users.rb b/cookbooks/midterm-vuln/attributes/users.rb new file mode 100644 index 0000000..703f47a --- /dev/null +++ b/cookbooks/midterm-vuln/attributes/users.rb @@ -0,0 +1,16 @@ +# +# Cookbook:: metasploitable +# Attributes:: users +# + + + +default[:users][:tyler] = { username: 'tyler', + password: 'redacted', + password_hash: '$6$n67I5.deJtxrC6qR$nFKbkISjdBZDJ1NW2c47zsSUPfeHdqr6rhDYy7SZVEENT8JrmXX6U5dQ8d2MizYpCMg9QBN2grtijnVuD1kDW/', + ssh_keys: '' + first_name: 'Tyler', + last_name: 'Hydra', + admin: true + } + diff --git a/cookbooks/midterm-vuln/files/sshd/sshd_config b/cookbooks/midterm-vuln/files/sshd/sshd_config new file mode 100644 index 0000000..783c567 --- /dev/null +++ b/cookbooks/midterm-vuln/files/sshd/sshd_config @@ -0,0 +1,88 @@ +# Package generated configuration file +# See the sshd_config(5) manpage for details + +# What ports, IPs and protocols we listen for +Port 22 +# Use these options to restrict which interfaces/protocols sshd will bind to +#ListenAddress :: +#ListenAddress 0.0.0.0 +Protocol 2 +# HostKeys for protocol version 2 +HostKey /etc/ssh/ssh_host_rsa_key +HostKey /etc/ssh/ssh_host_dsa_key +HostKey /etc/ssh/ssh_host_ecdsa_key +HostKey /etc/ssh/ssh_host_ed25519_key +#Privilege Separation is turned on for security +UsePrivilegeSeparation yes + +# Lifetime and size of ephemeral version 1 server key +KeyRegenerationInterval 3600 +ServerKeyBits 1024 + +# Logging +SyslogFacility AUTH +LogLevel INFO + +# Authentication: +LoginGraceTime 120 +PermitRootLogin without-password +StrictModes yes + +RSAAuthentication yes +PubkeyAuthentication yes +#AuthorizedKeysFile %h/.ssh/authorized_keys + +# Don't read the user's ~/.rhosts and ~/.shosts files +IgnoreRhosts yes +# For this to work you will also need host keys in /etc/ssh_known_hosts +RhostsRSAAuthentication no +# similar for protocol version 2 +HostbasedAuthentication no +# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication +#IgnoreUserKnownHosts yes + +# To enable empty passwords, change to yes (NOT RECOMMENDED) +PermitEmptyPasswords no + +# Change to yes to enable challenge-response passwords (beware issues with +# some PAM modules and threads) +ChallengeResponseAuthentication no + +# Change to no to disable tunnelled clear text passwords +PasswordAuthentication yes + +# Kerberos options +#KerberosAuthentication no +#KerberosGetAFSToken no +#KerberosOrLocalPasswd yes +#KerberosTicketCleanup yes + +# GSSAPI options +#GSSAPIAuthentication no +#GSSAPICleanupCredentials yes + +X11Forwarding yes +X11DisplayOffset 10 +PrintMotd no +PrintLastLog yes +TCPKeepAlive yes +#UseLogin no + +#MaxStartups 10:30:60 +#Banner /etc/issue.net + +# Allow client to pass locale environment variables +AcceptEnv LANG LC_* + +Subsystem sftp /usr/lib/openssh/sftp-server + +# Set this to 'yes' to enable PAM authentication, account processing, +# and session processing. If this is enabled, PAM authentication will +# be allowed through the ChallengeResponseAuthentication and +# PasswordAuthentication. Depending on your PAM configuration, +# PAM authentication via ChallengeResponseAuthentication may bypass +# the setting of "PermitRootLogin without-password". +# If you just want the PAM account and session checks to run without +# PAM authentication, then enable this but set PasswordAuthentication +# and ChallengeResponseAuthentication to 'no'. +UsePAM yes diff --git a/cookbooks/midterm-vuln/files/sshguard/whitelist b/cookbooks/midterm-vuln/files/sshguard/whitelist new file mode 100644 index 0000000..0ce4b1b --- /dev/null +++ b/cookbooks/midterm-vuln/files/sshguard/whitelist @@ -0,0 +1,6 @@ +# To see more examples, please see +# /usr/share/doc/sshguard/examples/whitelistfile.example + +# Address blocks in CIDR notation +127.0.0.0/8 +192.168.10.100/32 \ No newline at end of file diff --git a/cookbooks/midterm-vuln/metadata.rb b/cookbooks/midterm-vuln/metadata.rb new file mode 100644 index 0000000..30bb2b3 --- /dev/null +++ b/cookbooks/midterm-vuln/metadata.rb @@ -0,0 +1,8 @@ +name 'midterm-vuln' +maintainer 'deargle' +maintainer_email 'dave@daveeargle.com' +license 'BSD-3-clause' +description 'Configures vuln midterm server' +long_description 'Configures vuln midterm server' +version '0.0.1' +depends 'apt' \ No newline at end of file diff --git a/cookbooks/midterm-vuln/recipes/apt_update.rb b/cookbooks/midterm-vuln/recipes/apt_update.rb new file mode 100644 index 0000000..2b8e6c7 --- /dev/null +++ b/cookbooks/midterm-vuln/recipes/apt_update.rb @@ -0,0 +1,3 @@ +execute 'apt-get update' do + command 'apt-get update' +end \ No newline at end of file diff --git a/cookbooks/midterm-vuln/recipes/clear_cache.rb b/cookbooks/midterm-vuln/recipes/clear_cache.rb new file mode 100644 index 0000000..4f05322 --- /dev/null +++ b/cookbooks/midterm-vuln/recipes/clear_cache.rb @@ -0,0 +1,15 @@ +# +# Cookbook:: metasploitable +# Recipe:: clear_cache +# +# Copyright:: 2017, Rapid7, All Rights Reserved. + +directory '/var/chef' do + action :delete + recursive true +end + +directory '/vagrant' do + action :delete + recursive true +end diff --git a/cookbooks/midterm-vuln/recipes/default.rb b/cookbooks/midterm-vuln/recipes/default.rb new file mode 100644 index 0000000..e69de29 diff --git a/cookbooks/midterm-vuln/recipes/openvpn.rb b/cookbooks/midterm-vuln/recipes/openvpn.rb new file mode 100644 index 0000000..b524a93 --- /dev/null +++ b/cookbooks/midterm-vuln/recipes/openvpn.rb @@ -0,0 +1,18 @@ +apt_repository 'openvpn' do + uri "http://build.openvpn.net/debian/openvpn/release/2.4" + key "https://swupdate.openvpn.net/repos/repo-public.gpg" + components ["main"] +end + +package 'openvpn' + +bash 'add client autostart to openvpn' do + code <<-EOS + echo 'AUTOSTART="all"' >> /etc/default/openvpn + EOS + not_if 'grep -q AUTOSTART="all" /etc/default/openvpn' +end + +service 'openvpn' do + action [:enable, :start, :reload] +end \ No newline at end of file diff --git a/cookbooks/midterm-vuln/recipes/system_config.rb b/cookbooks/midterm-vuln/recipes/system_config.rb new file mode 100644 index 0000000..34d5bcc --- /dev/null +++ b/cookbooks/midterm-vuln/recipes/system_config.rb @@ -0,0 +1,12 @@ +execute 'set vim default editor' do + command 'update-alternatives --set editor /usr/bin/vim.basic' +end + +cookbook_file '/etc/sshguard/whitelist' do + source 'sshguard/whitelist' + mode '644' +end + +service 'sshguard' do + action [:restart] +end \ No newline at end of file diff --git a/cookbooks/midterm-vuln/recipes/users.rb b/cookbooks/midterm-vuln/recipes/users.rb new file mode 100644 index 0000000..5a730ad --- /dev/null +++ b/cookbooks/midterm-vuln/recipes/users.rb @@ -0,0 +1,84 @@ +# +# Cookbook:: midterm-vuln +# Recipe:: users +# +def keys_from_url(url) + host = url.split('/')[0..2].join('/') + path = url.split('/')[3..-1].join('/') + begin + response = Chef::HTTP.new(host).get(path) + response.split("\n") + rescue Net::HTTPServerException => e + p "request: #{host}#{path}, error: #{e}" + end +end + + +uid = 1111 + +node[:users].each do |u, attributes| + + home_dir = "/home/#{attributes[:username]}" + gid = uid + + group attributes[:username] do + gid gid + end + + user attributes[:username] do + manage_home true + password attributes[:password_hash] + uid uid + gid gid + home home_dir + shell '/bin/bash' + end + + # Add public key, if present in attributes + # Lifted from https://github.com/chef-cookbooks/users + if attributes[:ssh_keys] + + ssh_keys = [] + Array(attributes[:ssh_keys]).each do |key| + if key.start_with?('https') + ssh_keys += keys_from_url(key) + else + ssh_keys << key + end + end + + directory "#{home_dir}/.ssh" do + recursive true + owner uid + group gid + mode '0700' + end + + # use the keyfile defined in the attributes or fallback to the standard file in the home dir + key_file = attributes[:authorized_keys_file] || "#{home_dir}/.ssh/authorized_keys" + + template key_file do # ~FC022 + source 'authorized_keys.erb' + owner uid + group uid + mode '0600' + sensitive true + # ssh_keys should be a combination of u['ssh_keys'] and any keys + # returned from a specified URL + variables ssh_keys: ssh_keys + end + end + + + uid += 1 +end + +administrator_members = node[:users].keys.find_all { |user| node[:users][user][:admin] == true } + +group 'sudo' do + action :modify + members administrator_members.map { |u| node[:users][u][:username] } + append true +end + + diff --git a/cookbooks/midterm-vuln/templates/authorized_keys.erb b/cookbooks/midterm-vuln/templates/authorized_keys.erb new file mode 100644 index 0000000..96e9c30 --- /dev/null +++ b/cookbooks/midterm-vuln/templates/authorized_keys.erb @@ -0,0 +1,3 @@ +<% Array(@ssh_keys).each do |key| %> +<%= key %> +<% end -%> \ No newline at end of file diff --git a/main.tf b/main.tf new file mode 100644 index 0000000..d97ede9 --- /dev/null +++ b/main.tf @@ -0,0 +1,222 @@ +locals { + gcp_regions = ["us-central1","us-east1","us-east4","us-west1","us-west2"] + + team_number_split = split("-",terraform.workspace) + team_number = length(local.team_number_split) > 1 ? local.team_number_split[1] : 1 + google_region = local.gcp_regions[(local.team_number - 1) % length(local.gcp_regions)] +} + +provider "google" { + credentials = file("${var.gcp_service_account_credentials}") + + project = "midterm-vuln" + region = "${local.google_region}" + zone = "${local.google_region}-c" +} + +resource "google_compute_network" "vpc_network" { + name = "midterm-vuln-network-${terraform.workspace}" + auto_create_subnetworks = false +} + +resource "google_compute_subnetwork" "humbleify_subnet" { + name = "humbleify-subnet-${terraform.workspace}" + ip_cidr_range = "192.168.10.0/24" + region = "${local.google_region}" + network = "${google_compute_network.vpc_network.name}" +} + +resource "google_compute_firewall" "allow-ssh" { + name = "allow-ssh-${terraform.workspace}" + network = "${google_compute_network.vpc_network.name}" + direction = "INGRESS" + + allow { + protocol = "tcp" + ports = ["22"] + } +} + +resource "google_compute_firewall" "allow-icmp" { + name = "allow-icmp-${terraform.workspace}" + network = "${google_compute_network.vpc_network.name}" + direction = "INGRESS" + allow { + protocol = "icmp" + } +} + +resource "google_compute_firewall" "allow-all-internal" { + name = "allow-all-internal-${terraform.workspace}" + network = "${google_compute_network.vpc_network.name}" + direction = "INGRESS" + source_ranges = ["${google_compute_subnetwork.humbleify_subnet.ip_cidr_range}"] + + allow { + protocol = "tcp" + } + allow { + protocol = "udp" + } + allow { + protocol = "icmp" + } +} + +resource "google_compute_firewall" "allow-all-egress" { + name = "allow-all-egress-${terraform.workspace}" + network = "${google_compute_network.vpc_network.name}" + direction = "EGRESS" + + allow { + protocol = "tcp" + } + allow { + protocol = "udp" + } + allow { + protocol = "icmp" + } +} + +resource "google_compute_firewall" "allow-vpn-ingress" { + name = "allow-vpn-ingress-${terraform.workspace}" + network = "${google_compute_network.vpc_network.name}" + direction = "INGRESS" + + allow { + protocol = "udp" + ports = ["1194"] + } + + target_tags = ["openvpn"] +} + +data "google_compute_image" "midterm-vuln" { + family = "midterm-vuln" +} + +resource "google_compute_instance" "midterm-vuln" { + name = "humbleify-${terraform.workspace}" + machine_type = "g1-small" + allow_stopping_for_update = true + + tags = [terraform.workspace] + boot_disk { + initialize_params { + image = "${data.google_compute_image.midterm-vuln.self_link}" + } + } + + network_interface { + network = google_compute_network.vpc_network.name + subnetwork = google_compute_subnetwork.humbleify_subnet.name + network_ip = "192.168.10.107" + access_config {} + } +} + +resource "google_compute_instance" "openvpn" { + name = "openvpn-${terraform.workspace}" + machine_type = "f1-micro" + tags = ["openvpn", terraform.workspace] + + boot_disk { + initialize_params { + image = "debian-cloud/debian-9" + } + } + + network_interface { + network = google_compute_network.vpc_network.name + subnetwork = google_compute_subnetwork.humbleify_subnet.name + network_ip = "192.168.10.100" + access_config {} + } +} + +locals { + openvpn_public_ip = "${google_compute_instance.openvpn.network_interface.0.access_config.0.nat_ip}" + midterm_vuln_ip = "${google_compute_instance.midterm-vuln.network_interface.0.access_config.0.nat_ip}" + vpnconfig_user_filename = "client-${terraform.workspace}.conf" + vpnconfig_vulnserver_filename = "vulnserver-${terraform.workspace}.conf" +} + +resource "null_resource" "openvpn_bootstrap" { + connection { + type = "ssh" + host = local.openvpn_public_ip + user = "${var.ssh_username}" + port = "22" + private_key = "${file("${path.module}/${var.ssh_private_key_file}")}" + agent = false + } + + provisioner "remote-exec" { + inline = [ + "sudo apt update -y", + "curl -O ${var.openvpn_install_script_location}", + "chmod +x openvpn-install.sh", + "sudo AUTO_INSTALL=y DNS=13 ./openvpn-install.sh", + ] + } + provisioner "remote-exec" { + script = "${path.module}/scripts/bootstrap-openvpn.sh" + } + + provisioner "local-exec" { + command = <> ${local.vpnconfig_vulnserver_filename}; + echo "route 10.8.0.0 255.255.255.0" >> ${local.vpnconfig_vulnserver_filename}; + cd -; + EOT + } + +} + +resource "null_resource" "openvpn_upload_midtermvuln_vpn_config" { + depends_on = ["null_resource.openvpn_bootstrap"] + + triggers = { + # always_run = "${timestamp()}" + midterm_box = google_compute_instance.midterm-vuln.id + } + + connection { + type = "ssh" + host = local.midterm_vuln_ip + user = "${var.ssh_username}" + port = "22" + private_key = "${file("${path.module}/${var.ssh_private_key_file}")}" + agent = false + } + + provisioner "file" { + source = "${var.vpn_config_dir}/${local.vpnconfig_vulnserver_filename}" + destination = "~/${local.vpnconfig_vulnserver_filename}" + } + + provisioner "remote-exec" { + inline = [ + "sudo mv ~/${local.vpnconfig_vulnserver_filename} /etc/openvpn/", + "sudo service openvpn restart" + ] + } +} + +output "midterm_vuln_ip" { + value = local.midterm_vuln_ip +} + +output "team-name" { + value = terraform.workspace +} diff --git a/playbook-chef-solo.yml b/playbook-chef-solo.yml new file mode 100644 index 0000000..f63f6f1 --- /dev/null +++ b/playbook-chef-solo.yml @@ -0,0 +1,28 @@ +--- +- hosts: all + remote_user: _provisioner + become: yes + become_method: sudo + vars: + runlist: "{{ lookup('file','chef_runlist') }}" + tasks: + - name: Create temp chef dir + file: + path: /tmp/chef + state: directory + - name: Template and copy dna.json + template: + src: templates/dna.json.j2 + dest: /tmp/chef/dna.json + - name: Copy and Unzip cookbooks + unarchive: + src: chef-solo.tar.gz + dest: /tmp/chef/ + - name: Run chef solo + shell: chef-solo -j /tmp/chef/dna.json --config-option cookbook_path="['/tmp/chef/cookbooks','/tmp/chef/berks-cookbooks']" + register: out + - name: delete chef dir + file: + path: /tmp/chef + state: absent + - debug: var=out.stdout_lines diff --git a/scripts/ansible-commands b/scripts/ansible-commands new file mode 100644 index 0000000..3189a4f --- /dev/null +++ b/scripts/ansible-commands @@ -0,0 +1,3 @@ +ansible -i vuln_ips all -m ping --private-key tyler-midterm-vuln -u _provisioner -f 10 +ansible-playbook -i vuln_ips --private-key=tyler-midterm-vuln run-chef-solo.yml -f 10 +ansible-playbook -i vuln_ips -l team-999 playbook-chef-solo.yml diff --git a/scripts/ansible-playbook-999.sh b/scripts/ansible-playbook-999.sh new file mode 100755 index 0000000..c877eb7 --- /dev/null +++ b/scripts/ansible-playbook-999.sh @@ -0,0 +1,3 @@ +tar czvf chef-solo.tar.gz ./cookbooks ./berks-cookbooks +ansible-playbook -i vuln_ips -l team-999 --private-key=tyler-midterm-vuln playbook-chef-solo.yml -f 10 +rm chef-solo.tar.gz diff --git a/scripts/ansible-playbook.sh b/scripts/ansible-playbook.sh new file mode 100755 index 0000000..f21b038 --- /dev/null +++ b/scripts/ansible-playbook.sh @@ -0,0 +1,4 @@ +rm chef-solo.tar.gz +tar czvf chef-solo.tar.gz ./cookbooks ./berks-cookbooks +ansible-playbook -i vuln_ips --private-key=tyler-midterm-vuln playbook-chef-solo.yml -f 10 +rm chef-solo.tar.gz diff --git a/scripts/bootstrap-openvpn.sh b/scripts/bootstrap-openvpn.sh new file mode 100644 index 0000000..642ddd3 --- /dev/null +++ b/scripts/bootstrap-openvpn.sh @@ -0,0 +1,3 @@ +#!/bin/bash +echo 'push "route 192.168.10.0 255.255.255.0"' | sudo tee -a /etc/openvpn/server.conf +sudo systemctl restart openvpn@server \ No newline at end of file diff --git a/scripts/create_all.sh b/scripts/create_all.sh new file mode 100755 index 0000000..addeeed --- /dev/null +++ b/scripts/create_all.sh @@ -0,0 +1,15 @@ +#!/bin/bash +NUM_TEAMS=20 +START_FROM=1 +for i in $(seq $START_FROM $(( $START_FROM + $NUM_TEAMS )) ); do + WORKSPACE="team-${i}" + + # Select/Create Terraform Workspace + terraform workspace select "${WORKSPACE}" + IS_WORKSPACE_PRESENT=$? + if [ "${IS_WORKSPACE_PRESENT}" -ne "0" ] + then + terraform workspace new "${WORKSPACE}" + fi + TF_WORKSPACE=$WORKSPACE terraform apply -auto-approve +done diff --git a/scripts/destroy_all.sh b/scripts/destroy_all.sh new file mode 100755 index 0000000..15cf8ae --- /dev/null +++ b/scripts/destroy_all.sh @@ -0,0 +1,6 @@ +#!/bin/bash +TEAMS=$(terraform workspace list | sed 's:^..::') +for TEAM in $TEAMS ; do + TF_WORKSPACE=$TEAM terraform destroy -auto-approve +#TF_WORKSPACE=$WORKSPACE terraform workspace delete +done diff --git a/scripts/get_all_ips.sh b/scripts/get_all_ips.sh new file mode 100755 index 0000000..298c8ed --- /dev/null +++ b/scripts/get_all_ips.sh @@ -0,0 +1,8 @@ +#!/bin/bash +TEAMS=$(terraform workspace list | sed 's:^..::') +> vuln_ips +for TEAM in $TEAMS; do + echo "[$TEAM]" >> vuln_ips + TF_WORKSPACE=$TEAM terraform output midterm_vuln_ip >> vuln_ips + echo >> vuln_ips +done diff --git a/scripts/refresh_all.sh b/scripts/refresh_all.sh new file mode 100755 index 0000000..37f0a59 --- /dev/null +++ b/scripts/refresh_all.sh @@ -0,0 +1,5 @@ +#!/bin/bash +TEAMS=$(terraform workspace list | sed 's:^..::') +for TEAM in $TEAMS; do + TF_WORKSPACE=$TEAM terraform refresh +done diff --git a/scripts/update_all.sh b/scripts/update_all.sh new file mode 100755 index 0000000..c097711 --- /dev/null +++ b/scripts/update_all.sh @@ -0,0 +1,6 @@ +#!/bin/bash +TEAMS=$(terraform workspace list | sed 's:^..::') +for TEAM in $TEAMS; do + WORKSPACE="$TEAM" + TF_WORKSPACE=$WORKSPACE terraform apply -auto-approve +done diff --git a/scripts/update_all_reverse.sh b/scripts/update_all_reverse.sh new file mode 100755 index 0000000..93c9d13 --- /dev/null +++ b/scripts/update_all_reverse.sh @@ -0,0 +1,6 @@ +#!/bin/bash +TEAMS=$(terraform workspace list | sed 's:^..::' | tac | awk 'NF') +for TEAM in $TEAMS; do + WORKSPACE="$TEAM" + TF_WORKSPACE=$WORKSPACE terraform apply -auto-approve +done diff --git a/templates/dna.json.j2 b/templates/dna.json.j2 new file mode 100644 index 0000000..22921d2 --- /dev/null +++ b/templates/dna.json.j2 @@ -0,0 +1,3 @@ +{ +"run_list": [ {{runlist}} ] +} \ No newline at end of file diff --git a/variables.tf b/variables.tf new file mode 100644 index 0000000..9576f35 --- /dev/null +++ b/variables.tf @@ -0,0 +1,24 @@ +variable "ssh_username" { + description = "The user to connect to the instances as, typically the provisioner" + default = "_provisioner" +} + +variable "ssh_private_key_file" { + description = "" + default = "tyler-midterm-vuln" +} + +variable "gcp_service_account_credentials" { + default = "midterm-vuln-gcp-private-key.json" +} + +variable "ovpn_config_directory" { + description = "The name of the directory to eventually download the OVPN configuration files to" + default = "vpn_configs" +} + +variable "openvpn_install_script_location" { + description = "The location of an OpenVPN installation script compatible with https://raw.githubusercontent.com/angristan/openvpn-install/master/openvpn-install.sh" + default = "https://raw.githubusercontent.com/deargle/openvpn-install/master/openvpn-install.sh" +} + diff --git a/vpn_configs/.gitkeep b/vpn_configs/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/vuln_ips b/vuln_ips new file mode 100644 index 0000000..4b3800b --- /dev/null +++ b/vuln_ips @@ -0,0 +1 @@ +# this file will be populated by `scripts/get_all_ips.sh` \ No newline at end of file