diff --git a/docs/applications/configuration-management/install-a-chef-server-workstation-on-ubuntu-18-04/index.md b/docs/applications/configuration-management/install-a-chef-server-workstation-on-ubuntu-18-04/index.md index 8814bddf332..0a2da35024a 100644 --- a/docs/applications/configuration-management/install-a-chef-server-workstation-on-ubuntu-18-04/index.md +++ b/docs/applications/configuration-management/install-a-chef-server-workstation-on-ubuntu-18-04/index.md @@ -3,22 +3,22 @@ author: name: Linode email: docs@linode.com description: 'Instructions on how to configure a Chef server and a virtual workstation, and how to bootstrap a client node on Ubuntu 18.04.' -keywords: ["chef", "chef installation", "configuration change management", "server automation", "chef server", "chef workstation", "chef-client", "knife.rb", "version control"] +keywords: ["chef", "configuration management", "server automation", "chef server", "chef workstation", "chef-client"] license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' published: 2018-08-06 -modified: 2018-08-06 +modified: 2019-01-17 modified_by: name: Linode title: 'Install a Chef Server Workstation on Ubuntu 18.04' --- -[Chef](http://www.chef.io) is an automation platform that "turns infrastructure into code," allowing users to manage and deploy resources across multiple servers, or *nodes*. Chef allows users to create and download recipes (stored in *cookbooks*) to automate content and policies on these nodes. - -Chef is comprised of a Chef server, one or more workstations, and a number of nodes that are managed by the chef-client installed on each node. +[Chef](http://www.chef.io) is a Ruby based configuration management tool used to define infrastructure as code. This enables users to automate the management of many *nodes* and maintain consistency across those nodes. *Recipes* declare the desired state for managed nodes and are created on a user's *workstation* using the *Chef Workstation* package. Your recipes are distributed across nodes via a *Chef server*. A *Chef client*, installed on each node, is in charge of applying the recipe to its corresponding node. [![chef_graph-small.png](chef-graph-small.png)](chef-graph.png "Illustration showing that workstations talk to Nodes via the Chef Server") -This guide shows how to create and configure a Chef server, a virtual workstation, and how to bootstrap a node to run the chef-client, all on individual Linodes. +This guide will show you how to create and configure a Chef server and workstation. You will also bootstrap a node to manage with Chef. This work will require three individual Linodes. + +See [A Beginner's Guide to Chef](/docs/applications/configuration-management/beginners-guide-chef/) for an introduction to Chef concepts. {{< note >}} This guide is written for a non-root user. Commands that require elevated privileges are prefixed with `sudo`. If you're not familiar with the `sudo` command, you can check our [Users and Groups](/docs/tools-reference/linux-users-and-groups/#understanding-sudo) guide. @@ -26,23 +26,24 @@ This guide is written for a non-root user. Commands that require elevated privil ## Prerequisites -- One 4GB Linode, running Ubuntu 18.04, to host the Chef server -- Two Linodes of any size, each running Ubuntu 18.04, to host a workstation and a node -- Each Linode should be configured per the [Getting Started](/docs/getting-started/) and [Securing Your Sever](/docs/security/securing-your-server/) guides -- Each Linode needs to be configured with a valid FQDN +- One 8GB Linode running Ubuntu 18.04. This Linode will host the Chef server. + - Assign a Domain to the Chef server. Ensure your domain has a corresponding domain zone, NS record, and A/AAA record. See the [DNS Manager guide](/docs/platform/manager/dns-manager-new-manager/#add-a-domain-zone) for details. + - Ensure your Chef server's hostname is the same as its Domain name. Your Chef server will automatically create SSL certificates based on the Linode's hostname. +- Two 2 GB Linodes, each running Ubuntu 18.04. One Linode will host a workstation and the other a node to be managed by Chef. +- The workstation and Chef server should be configured per the [Getting Started](/docs/getting-started/) and [Securing Your Sever](/docs/security/securing-your-server/) guides. Once your node is [bootstrapped](/docs/applications/configuration-management/install-a-chef-server-workstation-on-ubuntu-18-04/#bootstrap-a-node), you can use a Chef cookbook to secure your node. Consider using the [Users](https://supermarket.chef.io/cookbooks/users) cookbook and the [Firewall](https://supermarket.chef.io/cookbooks/firewall) cookbook for this work. - Ensure that all servers are up-to-date: sudo apt update && sudo apt upgrade ## The Chef Server -The Chef server is the hub of interaction between all workstations and nodes using Chef. Changes made through workstations are uploaded to the Chef server, which is then accessed by `chef-client` and used to configure each individual node. +The Chef server is the hub of interaction between all workstations and nodes under Chef management. Changes made to configuration code on workstations are pushed to the Chef server, and then pulled by a node's `chef-client` to apply the configurations. ### Install the Chef Server 1. Download the [latest Chef server core](https://downloads.chef.io/chef-server/#ubuntu): - wget https://packages.chef.io/files/current/chef-server/12.17.54+20180531095715/ubuntu/18.04/chef-server-core_12.17.54+20180531095715-1_amd64.deb + wget https://packages.chef.io/files/stable/chef-server/12.18.14/ubuntu/18.04/chef-server-core_12.18.14-1_amd64.deb 1. Install the server: @@ -68,50 +69,78 @@ In order to link workstations and nodes to the Chef server, create an administra sudo chef-server-ctl user-create USER_NAME FIRST_NAME LAST_NAME EMAIL 'PASSWORD' --filename ~/.chef/USER_NAME.pem + To view a list of all users on your Chef server issue the following command: + + sudo chef-server-ctl user-list + 1. Create an organization and add the user created in the previous step to the admins and billing admins security groups. Replace `ORG_NAME` with a short identifier for the organization, `ORG_FULL_NAME` with the organizations' complete name, `USER_NAME` with the username created in the step above and `ORG_NAME.pem` with organization's short identifier followed by `.pem`: sudo chef-server-ctl org-create ORG_NAME "ORG_FULL_NAME" --association_user USER_NAME --filename ~/.chef/ORG_NAME.pem - With the Chef server installed and the RSA keys generated, you can begin configuring your workstation, where all major work will be performed for your Chef's nodes. + To view a list of all organizations on your Chef server, use the following command: + + sudo chef-server-ctl org-list + +With the Chef server installed and the RSA keys generated, you can begin configuring your workstation. The workstation is where all major configurations will be created for your nodes. ## Chef Workstations -Your Chef workstation is where you create and configure any recipes, cookbooks, attributes, and other changes made to your Chef configurations. Although this can be a local machine running any OS, there is some benefit to keeping a remote server as your workstation so you can access it from anywhere. +The Chef workstation is where you create and configure any recipes, cookbooks, attributes, and other changes necessary to manage your nodes. Although this can be a local machine running any OS, there is some benefit to keeping a remote server as your workstation so you can access it from anywhere. + +In this section, you will download and install the Chef Workstation package, which provides all tools also included with the [ChefDK](https://docs.chef.io/about_chefdk.html), Chef's development kit. ### Setting Up a Workstation -1. [Download the latest Chef Development Kit](https://downloads.chef.io/chef-dk/ubuntu/): +1. [Download the latest Chef Workstation](https://downloads.chef.io/chef-workstation/0.2.43#ubuntu): - wget https://packages.chef.io/files/stable/chefdk/3.1.0/ubuntu/18.04/chefdk_3.1.0-1_amd64.deb + wget https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb -1. Install ChefDK: +1. Install Chef Workstation: - sudo dpkg -i chefdk_*.deb + dpkg -i chef-workstation_*.deb 1. Remove the installation file: - rm chefdk_*.deb + rm chef-workstation_*.deb -1. Generate the `chef-repo` and move into the newly-created directory: +1. Create your Chef repository. The `chef-repo` directory will store your Chef cookbooks and other related files. - chef generate app chef-repo - cd chef-repo + chef generate repo chef-repo -1. Make the `.chef` directory to store the `knife.rb`, `ORGANIZATION-validator.pem`, and `USER.pem` files later: +1. Ensure that your workstation's `/etc/hosts` file maps its IP address to your workstation's hostname. For example: - mkdir .chef + {{< file "/etc/hosts">}} +127.0.0.1 localhost +192.0.2.0 workstation +... + {{}} + +1. Create a `.chef` subdirectory. The `.chef` subdirectory will store your [Knife](/docs/applications/configuration-management/beginners-guide-chef/#knife) configuration file and your `.pem` files that are used for RSA key pair authentication with the Chef server. Move into the `chef-repo` directory: + + mkdir ~/chef-repo/.chef + cd chef-repo ### Add the RSA Private Keys -1. The RSA private keys generated when setting up the Chef server now need to be placed on the workstation. The process behind this will vary depending on whether or not you are using SSH key pair authentication to log into your Linodes. +Authentication between the Chef server and workstation and/or nodes is completed with public key encryption. This ensures that the Chef server only communicates with trusted machines. In this section, the RSA private keys, generated when setting up the Chef server, will be copied to the workstation to allow communicate between the Chef server and workstation. + +1. If you do not already have an RSA key-pair on your workstation, generate one. This key-pair will be used to gain access to the Chef server and then transfer their `.pem` files: - - **If you are not using key pair authentication**, copy the file directly from the Chef Server. Replace `user` with your username on the server, and `192.0.2.0` with the URL or IP of your Chef Server: + ssh-keygen -b 4096 - scp user@192.0.2.0:~/.chef/*.pem ~/chef-repo/.chef/ + Press **Enter** to use the default names `id_rsa` and `id_rsa.pub` in `/home/your_username/.ssh` before entering your passphrase. - - **If you are using key pair authentication**, then from your **local terminal** copy the `.pem` files from your server to your workstation using the `scp` command. Replace `user` with the appropriate username, and `192.0.2.0` with the URL or IP for your Chef Server and `203.0.113.0` with the URL or IP for your workstation: + {{< note >}} + If you have disabled SSH password authentication on your Chef server's Linode, as recommended by the [How to Secure Your Server](/docs/security/securing-your-server/#ssh-daemon-options) guide, re-enable SSH password authentication prior to performing these steps. Be sure to disable it again once you have added your workstation's public ssh key to the Chef server's Linode. + {{}} - scp -3 user@192.0.2.0:~/.chef/*.pem user@203.0.113.0:~/chef-repo/.chef/ + Upload your workstation's public key to the Linode hosting the Chef server. Ensure you replace `example_user` with the Chef server's user account and `192.0.2.0` with its IP address: + + ssh-copy-id example_user@192.0.2.0 + +1. Copy the `.pem` files from your Chef server to your workstation using the `scp` command. Replace `user` with the appropriate username, and `192.0.2.0` with your Chef server's IP: + + scp example_user@192.0.2.0:~/.chef/*.pem ~/chef-repo/.chef/ 1. Confirm that the files have been copied successfully by listing the contents of the `.chef` directory: @@ -121,23 +150,24 @@ Your Chef workstation is where you create and configure any recipes, cookbooks, ### Add Version Control -The workstation is used to add and edit cookbooks and other configuration files. It is beneficial to implement some form of version control. The ChefDK adds the Git component to your workstation and initializes a Git repository in the directory used to generate the Chef repo. Configure Git to add your username and email, and to add and commit any new files generated in the steps above. +The workstation is used to create, download, and edit cookbooks and other related files. You should track any changes made to these files with version control software, like Git. The Chef Workstation adds the Git component to your workstation and initializes a Git repository in the directory where the `chef-repo` was generated. Configure Git by adding your username and email, and add and commit any new files created in the steps above. 1. Configure Git by adding your username and email, replacing the needed values: git config --global user.name yourname git config --global user.email user@email.com -1. Add the `.chef` directory to the `.gitignore` file: +2. Add the `.chef` directory to the `.gitignore` file: - echo ".chef" > .gitignore + echo ".chef" > ~/chef-repo/.gitignore -1. Add and commit all existing files: +3. Move into the `~/chef-repo` directory, if you are not already there and add and commit all existing files: + cd ~/chef-repo git add . git commit -m "initial commit" -1. Make sure the directory is clean: +4. Make sure the directory is clean: git status @@ -148,13 +178,23 @@ The workstation is used to add and edit cookbooks and other configuration files. nothing to commit, working directory clean {{}} -### Generate knife.rb +## Generate your First Cookbook + +1. Generate a new Chef cookbook: -1. Create a knife configuration file by navigating to your `~/chef-repo/.chef` folder and creating a file named `knife.rb` in your chosen text editor. + chef generate cookbook my_cookbook +1. Generate the `chef-repo` and move into the newly-created directory: + + chef generate app chef-repo + cd chef-repo -1. Copy the following configuration into the `knife.rb` file: +### Configure Knife - {{< file "~/chef-repo/.chef/knife.rb" ruby >}} +1. Create a knife configuration file by navigating to your `~/chef-repo/.chef` directory and creating a file named `config.rb` using your preferred text editor. + +1. Copy the following configuration into the `config.rb` file: + + {{< file "~/chef-repo/.chef/config.rb" ruby >}} current_dir = File.dirname(__FILE__) log_level :info log_location STDOUT @@ -170,11 +210,11 @@ cookbook_path ["#{current_dir}/../cookbooks"] 1. Change the following: - - The value for `node_name` should be the username that was created on the chef server. + - The value for `node_name` should be the username that was created on the Chef server. - Change `USER.pem` under `client_key` to reflect your `.pem` file for your user. - The `validation_client_name` should be your organization's `ORG_NAME` followed by `-validator`. - `ORGANIZATION-validator.pem` in the `validation_key` path should be set to the `ORG_NAME` followed by `-validator.pem`. - - Finally the `chef_server_url` needs to contain the IP address or FQDN of your Chef server, with your `ORG_NAME`. + - Finally the `chef_server_url` should be the Chef server's domain with `/organizations/ORG_NAME` appended. Be sure to replace `ORG_NAME` with your own organization's name. 1. Move to the `chef-repo` directory and copy the needed SSL certificates from the server: @@ -185,17 +225,21 @@ cookbook_path ["#{current_dir}/../cookbooks"] The SSL certificates are generated during the installation of the Chef server. These certificates are self-signed, which means there isn’t a signing certificate authority (CA) to verify. The Chef server's hostname and FQDN should be the same so that the workstation can fetch and verify the SSL certificates. You can verify the Chef server's hostname and FQDN by running `hostname` and `hostname -f`, respectively. Consult the [Chef documentation](https://docs.chef.io/server_security.html#regenerate-certificates) for details on regenerating SSL certificates. {{}} -1. Confirm that `knife.rb` is set up correctly by running the client list: +1. Confirm that `config.rb` is set up correctly by running the client list: knife client list This command should output the validator name. -With both the server and a workstation configured, it is possible to bootstrap your first node. +Now that your Chef server and workstation are configured, you can bootstrap your first node. ## Bootstrap a Node -Bootstrapping a client node installs the Chef client and validates the node, allowing it to read from the Chef server and make any needed configuration changes picked up by the chef-client in the future. +Bootstrapping a node installs the Chef client on the node and validates the node. This allows the node to read from the Chef server and pull down and apply any needed configuration updates detected by the chef-client. + +{{< note >}} +If you encounter any `401 Unauthorized` errors ensure that your `ORGANIZATION.pem` file has `700` permissions. See [Chef's troubleshooting](https://docs.chef.io/errors.html) guide for further information on diagnosing authentication errors. +{{}} 1. From your *workstation*, navigate to your `~/chef-repo/.chef` directory: @@ -215,17 +259,26 @@ Bootstrapping a client node installs the Chef client and validates the node, all knife bootstrap 192.0.2.0 --ssh-user username --sudo --identity-file ~/.ssh/id_rsa.pub --node-name hostname -2. Confirm that the node has been bootstrapped by listing the client nodes: +1. Confirm that the node has been bootstrapped by listing the client nodes: knife client list Your new client node should be included in the list. +1. Add the bootstrapped node to your workstation's `/etc/hosts` file. Replace `node-hostname` with the hostname you just assigned to the node when it was bootstrapped: + + {{< file "/etc/hosts">}} +127.0.0.1 localhost +192.0.2.0 workstation +198.51.100.0 node-hostname +... + {{}} + ## Download a Cookbook (Optional) -When using Chef you will want the Chef client to periodically run on your nodes and pull in any changes pushed to the Chef server. You will also want the `validation.pem` file that is uploaded to your node upon bootstrap to be deleted for security purposes. While these steps can be performed manually, it is often easier and more efficient to set it up as a cookbook. +When using Chef, the Chef client should periodically run on your nodes and pull down any changes pushed to the Chef server from your workstation. You will also want the `validation.pem` file that is uploaded to your node upon bootstrap to be deleted for security purposes. While these steps can be performed manually, it is often easier and more efficient to set them up as a cookbook. -This section is optional, but provides instructions on downloading a cookbook to your workstation, pushing it to a server, and includes the skeleton of a basic cookbook to expand and experiment with. +This section is optional, but provides instructions on downloading a cookbook to your workstation and pushing it to a server, and includes the skeleton of a basic cookbook to expand on and experiment with. 1. From your *workstation*, navigate to your `~/chef-repo/.chef` directory: @@ -260,7 +313,7 @@ end `file "/etc/chef/validation.pem" do` calls to the `validation.pem` file. The `action` defines that the file should be removed (`:delete`). - These are two very basic sets of code in Ruby, and provide an example of the code structure that will be used when creating Chef cookbooks. These examples can be edited and expanded as needed. + These are two very basic sets of code written in Ruby that provide an example of the code structure that will be used when creating Chef cookbooks. These examples can be edited and expanded as needed. 1. Add the recipe to your node's run list, replacing `nodename` with your node's name: @@ -272,10 +325,10 @@ end This command is also used when updating cookbooks. -1. Switch to your *bootstrapped* node(s) and run the initial `chef-client` command: +1. Use `knife-ssh` to run the `chef-client` command on your node. Replace `nodename` with your node's name. If you have set up your node with a limited user account, replace `-x root` with the correct username, i.e. `-x username`. - chef-client + knife ssh 'name:nodename' 'sudo chef-client' -x root - If running the node as a non-root user, append the above command with `sudo`. + The recipes in the run list will be pulled from the server and run on the node. In this instance, it will be the `cron-delvalidate` recipe. This recipe ensures that any cookbooks pushed to the Chef Server, and added to the node's run list will be pulled down to bootstrapped nodes once an hour. This automated step eliminates connecting to the node in the future to pull down changes. - The recipes in the run list will be pulled from the server and run. In this instance, it will be the `cron-delvalidate` recipe. This recipe ensures that any cookbooks made, pushed to the Chef Server, and added to the node's run list will be pulled down to bootstrapped nodes once an hour. This automated step eliminates connecting to the node in the future to pull down changes. +Now that you know how to download Chef cookbooks, add the cookbook to a node's run list, and apply the configurations on a node, consider securing your Chef managed node with the following cookbooks: [Users](https://supermarket.chef.io/cookbooks/users) and [Firewall](https://supermarket.chef.io/cookbooks/firewall).