Chef is a Ruby based configuration management tool used to define infrastructure as code. This enables users to automate the management of many nodes and maintain consistency across those nodes. Recipes declare the desired state for managed nodes and are created on a user’s workstation using the Chef Workstation package. Your recipes are distributed across nodes via a Chef server. A Chef client, installed on each node, is in charge of applying the recipe to its corresponding node. This guide will show you how to install and configure a Chef Server and Chef Workstation. We will also bootstrap a node to manage with Chef.
In this tutorial you will learn:
- Install and Configure the Chef Server
- Create Chef User and Organization
- Install and Configure the Chef Workstation
- Configure Knife and Bootstrapping a Client Node
Software Requirements and Conventions Used
| Category | Requirements, Conventions or Software Version Used |
|---|---|
| System | Ubuntu Linux 18.04 |
| Software | Chef Server Core, Chef Workstation, Chef Client, Chef Development Kit |
| Other | Privileged access to your Linux system as root or via the sudo command. |
| Conventions |
# – requires given linux commands to be executed with root privileges either directly as a root user or by use of sudo command$ – requires given linux commands to be executed as a regular non-privileged user |
Install and Configure the Chef Server
The Chef server is the hub of interaction between all workstations and nodes under Chef management. Changes made to configuration code on workstations are pushed to the Chef server, and then pulled by a node’s chef-client to apply the configurations.
Download the latest Chef server via running the below command:
# wget https://packages.chef.io/files/stable/chef-server/12.18.14/ubuntu/18.04/chef-server-core_12.18.14-1_amd64.deb
Now install the server package by the following command:
# dpkg -i chef-server-core_*.deb
The Chef server includes a command line utility called chef-server-ctl. Run chef-server-ctl to start the Chef server services.
# chef-server-ctl reconfigure
Upon successful configuration of the chef server you will see the below message and check the chef server running status as well.
Chef Client finished, 493/1100 resources updated in 12 minutes 02 seconds Chef Server Reconfigured!
root@ubuntubox1:~# chef-server-ctl status run: bookshelf: (pid 1435) 6644s; run: log: (pid 1440) 6644s run: nginx: (pid 1235) 6653s; run: log: (pid 1744) 6631s run: oc_bifrost: (pid 1196) 6657s; run: log: (pid 1203) 6657s run: oc_id: (pid 1220) 6655s; run: log: (pid 1227) 6655s run: opscode-erchef: (pid 4376) 6432s; run: log: (pid 1508) 6644s run: opscode-expander: (pid 1335) 6648s; run: log: (pid 1431) 6646s run: opscode-solr4: (pid 1244) 6650s; run: log: (pid 1285) 6649s run: postgresql: (pid 1176) 6659s; run: log: (pid 1180) 6659s run: rabbitmq: (pid 4188) 6443s; run: log: (pid 1748) 6631s run: redis_lb: (pid 27397) 6931s; run: log: (pid 1735) 6632s root@ubuntubox1:~#
Create Chef User and Organization
In order to link workstations and nodes to the Chef server, create an administrator and organization with associated RSA private keys.
From the home directory, create a .chef directory to store the keys.
# mkdir .chef
Use chef-server-ctl to create a user. In this example, change the following to match your needs: USER_NAME, FIRST_NAME, LAST_NAME, EMAIL, and PASSWORD. Adjust USER_NAME.pem, and leave the .pem extension.
chef-server-ctl user-create USER_NAME FIRST_NAME LAST_NAME EMAIL 'PASSWORD' --filename ~/.chef/USER_NAME.pem
root@ubuntubox1:~# chef-server-ctl user-create chefadmin Chef Administrator chefadmin@@ubuntubox1.com '*******' --filename ~/.chef/chefadmin.pem
To view the list of all users on your Chef server issue the following command:
root@ubuntubox1:~# chef-server-ctl user-list chefadmin pivotal
Create an organization and add the user created in the previous step. Replace ORG_NAME with a short identifier for the organization, ORG_FULL_NAME with the organizations’ complete name, USER_NAME with the username created in the step above and ORG_NAME.pem with organization’s short identifier followed by .pem.
chef-server-ctl org-create ORG_NAME "ORG_FULL_NAME" --association_user USER_NAME --filename ~/.chef/ORG_NAME.pem
root@ubuntubox1:~# chef-server-ctl org-create chef-on-ubuntu "Chef Infrastructure on Ubuntu 18.04" --association_user chefadmin --filename ~/.chef/chef-on-ubuntu.pem
To view the list of all organizations on your Chef server, use the following command:
root@ubuntubox1:~# chef-server-ctl org-list chef-on-ubuntu
With the Chef server installed and the RSA keys generated, we will begin configuring the Chef workstation. The workstation is where all major configurations will be created for your nodes.
Install and Configure the Chef Workstation
The Chef workstation is where you create and configure any recipes, cookbooks, attributes, and other changes necessary to manage your nodes. Although this can be a local machine running any OS, there is some benefit to keeping a remote server as your workstation so you can access it from anywhere.
In this section, you will download and install the Chef Workstation package, which provides all tools also included with the ChefDK, Chef’s development kit.
Download the latest Chef Workstation:
root@ubuntubox2:~# wget https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb --2019-06-03 13:35:51-- https://packages.chef.io/files/stable/chef-workstation/0.2.43/ubuntu/18.04/chef-workstation_0.2.43-1_amd64.deb Resolving packages.chef.io (packages.chef.io)... 151.101.142.110 Connecting to packages.chef.io (packages.chef.io)|151.101.142.110|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 129713682 (124M) [application/x-debian-package] Saving to: ‘chef-workstation_0.2.43-1_amd64.deb’ chef-workstation_0.2.43-1_amd64.deb 100%[=====================================================================================>] 123.70M 1.51MB/s in 80s 2019-06-03 13:37:17 (1.55 MB/s) - ‘chef-workstation_0.2.43-1_amd64.deb’ saved [129713682/129713682]
Install Chef Workstation:
root@ubuntubox2:~# dpkg -i chef-workstation_*.deb
Selecting previously unselected package chef-workstation.
(Reading database ... 117468 files and directories currently installed.)
Preparing to unpack chef-workstation_0.2.43-1_amd64.deb ...
Unpacking chef-workstation (0.2.43-1) ...
Setting up chef-workstation (0.2.43-1) ...
To run the experimental Chef Workstation App, use your
platform's package manager to install these dependencies:
libgconf-2.so.4 => not found
You can then launch the App by running 'chef-workstation-app'.
The App will then be available in the system tray.
Thank you for installing Chef Workstation!
You can find some tips on getting started at https://chef.sh/
Now we need to create the Chef repository. The chef-repo directory will store your Chef cookbooks and other related files.
# chef generate repo chef-repo
Create a .chef subdirectory. The .chef subdirectory will store the Knife configuration file and the .pem files that are used for RSA key pair authentication with the Chef server. Move into the chef-repo directory:
root@ubuntubox2:~# mkdir ~/chef-repo/.chef root@ubuntubox2:~# cd chef-repo root@ubuntubox2:~/chef-repo#
Authentication between the Chef server and workstation and/or nodes is completed with public key encryption. This ensures that the Chef server only communicates with trusted machines. In this section, the RSA private keys, generated when setting up the Chef server, will be copied to the workstation to allow communicate between the Chef server and workstation.
We will generate a RSA key-pair on the workstation server. This key-pair will be used to gain access to the Chef server and then transfer their .pem files:
root@ubuntubox2:~# ssh-keygen -b 4096 Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Created directory '/root/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: SHA256:sR+Nloq6vsc7rX4ZmMInP3SKdk4fYEJH1iLoKNm1YMg root@ubuntubox2.com The key's randomart image is: +---[RSA 4096]----+ |.. . o. | |.E+ oo. . | | * o.o... | |+ o... o + | |. .. ooS = . | | +o=oo+ . | | Oo+oo. | | ooO.+. | | o=B=*. | +----[SHA256]-----+ root@ubuntubox2:~#
Upload the workstation node’s public key to the Chef server node.
root@ubuntubox2:~# ssh-copy-id root@ubuntubox1.com /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@ubuntubox1.com's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@ubuntubox1.com'" and check to make sure that only the key(s) you wanted were added. root@ubuntubox2:~#
Copy the .pem files from your Chef server to your workstation using the scp command.
root@ubuntubox2:~# scp root@ubuntubox1.com:~/.chef/*.pem ~/chef-repo/.chef/ chefadmin.pem 100% 1674 27.9KB/s 00:00 chef-on-ubuntu.pem 100% 1674 496.8KB/s 00:00
Confirm that the files have been copied successfully by listing the contents of the .chef directory. The .pem files should be listed.
root@ubuntubox2:~# ls ~/chef-repo/.chef chefadmin.pem chef-on-ubuntu.pem
Generate a new Chef cookbook:
root@ubuntubox2:~# chef generate cookbook chef-first-cookbook Hyphens are discouraged in cookbook names as they may cause problems with custom resources. See https://docs.chef.io/ctl_chef.html#chef-generate-cookbook for more information. Generating cookbook chef-first-cookbook - Ensuring correct cookbook file content - Ensuring delivery configuration - Ensuring correct delivery build cookbook content Your cookbook is ready. Type `cd chef-first-cookbook` to enter it. There are several commands you can run to get started locally developing and testing your cookbook. Type `delivery local --help` to see a full list. Why not start by writing a test? Tests for the default recipe are stored at: test/integration/default/default_test.rb If you'd prefer to dive right in, the default recipe can be found at: recipes/default.rb
Generate the chef-repo and move into the newly-created directory:
# chef generate app chef-repo # cd chef-repo
Configure Knife and Bootstrapping a Client Node
Create a knife configuration file by navigating to ~/chef-repo/.chef directory and creating a file named config.rb using your preferred text editor.
Copy the following configuration into the config.rb file:
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name 'node_name'
client_key "USER.pem"
validation_client_name 'ORG_NAME-validator'
validation_key "ORGANIZATION-validator.pem"
chef_server_url 'https://ubuntubox1.com/organizations/ORG_NAME'
cache_type 'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path ["#{current_dir}/../cookbooks"]
The value for node_name should be the username that was created on the Chef server.
Change USER.pem under client_key to reflect the .pem file for chef user.
The validation_client_name should be the organization’s ORG_NAME followed by -validator.
ORGANIZATION-validator.pem in the validation_key path should be set to the ORG_NAME followed by -validator.pem.
Finally the chef_server_url should be the Chef server’s domain with /organizations/ORG_NAME appended. Be sure to replace ORG_NAME with the organization’s name.
Move to the chef-repo directory and copy the needed SSL certificates from the server:
root@ubuntubox2:~/chef-repo/.chef# cd .. root@ubuntubox2:~/chef-repo# knife ssl fetch WARNING: Certificates from ubuntubox1.com will be fetched and placed in your trusted_cert directory (/root/chef-repo/.chef/trusted_certs). Knife has no means to verify these are the correct certificates. You should verify the authenticity of these certificates after downloading. Adding certificate for ubuntubox1_com in /root/chef-repo/.chef/trusted_certs/ubuntubox1_com.crt
Confirm that config.rb is set up correctly by running the client list. This command should output the validator name.
root@ubuntubox2:~/chef-repo# knife client list chef-on-ubuntu-validator
Bootstrapping a node installs the Chef client on the node and validates the node. This allows the node to read from the Chef server and pull down and apply any needed configuration updates detected by the chef-client.
From the workstation, navigate to ~/chef-repo/.chef directory:
# cd ~/chef-repo/.chef
Bootstrap the client node either using the client node’s root user, or a user with elevated privileges:
root@ubuntubox2:~/chef-repo/.chef# knife bootstrap ubuntubox3.com -x root -P ******* --node-name chef-client-node Creating new client for chef-client-node Creating new node for chef-client-node Connecting to ubuntubox3.com ubuntubox3.com -----> Installing Chef Omnibus (-v 14) ubuntubox3.com downloading https://omnitruck-direct.chef.io/chef/install.sh ubuntubox3.com to file /tmp/install.sh.2019/install.sh ubuntubox3.com trying wget... ubuntubox3.com ubuntu 18.04 x86_64 ubuntubox3.com Getting information for chef stable 14 for ubuntu... ubuntubox3.com downloading https://omnitruck-direct.chef.io/stable/chef/metadata?v=14&p=ubuntu&pv=18.04&m=x86_64 ubuntubox3.com to file /tmp/install.sh.2023/metadata.txt ubuntubox3.com trying wget... ubuntubox3.com sha1 ed9b1fcdaf947d9a3d60e6d196308183a082bcff ubuntubox3.com sha256 9ddcd5ceef19c95ecc1f34bef080c23d9cb42ae8ebc69fd41dcf1c768a6a708f ubuntubox3.com url https://packages.chef.io/files/stable/chef/14.13.11/ubuntu/18.04/chef_14.13.11-1_amd64.deb ubuntubox3.com version 14.13.11 ubuntubox3.com downloaded metadata file looks valid... ubuntubox3.com downloading https://packages.chef.io/files/stable/chef/14.13.11/ubuntu/18.04/chef_14.13.11-1_amd64.deb ubuntubox3.com to file /tmp/install.sh.2023/chef_14.13.11-1_amd64.deb ubuntubox3.com trying wget... ubuntubox3.com Comparing checksum with sha256sum... ubuntubox3.com Installing chef 14 ubuntubox3.com installing with dpkg... ubuntubox3.com Selecting previously unselected package chef. (Reading database ... 117468 files and directories currently installed.) ubuntubox3.com Preparing to unpack .../chef_14.13.11-1_amd64.deb ... ubuntubox3.com Unpacking chef (14.13.11-1) ... ubuntubox3.com Setting up chef (14.13.11-1) ... ubuntubox3.com Thank you for installing Chef Infra Client! For help getting started visit https://learn.chef.io ubuntubox3.com Starting the first Chef Client run... ubuntubox3.com Starting Chef Client, version 14.13.11 ubuntubox3.com resolving cookbooks for run list: [] ubuntubox3.com Synchronizing Cookbooks: ubuntubox3.com Installing Cookbook Gems: ubuntubox3.com Compiling Cookbooks... ubuntubox3.com [2019-06-03T14:01:44+04:00] WARN: Node chef-client-node has an empty run list. ubuntubox3.com Converging 0 resources ubuntubox3.com ubuntubox3.com Running handlers: ubuntubox3.com Running handlers complete ubuntubox3.com Chef Client finished, 0/0 resources updated in 05 seconds Confirm that the node has been bootstrapped by listing the client nodes: root@ubuntubox2:~/chef-repo/.chef#
Confirm that the node has been bootstrapped successfully by listing the nodes:
root@ubuntubox2:~/chef-repo/.chef# knife node list chef-client-node root@ubuntubox2:~/chef-repo/.chef# knife node show chef-client-node Node Name: chef-client-node Environment: _default FQDN: ubuntubox3.com IP: 192.168.1.107 Run List: Roles: Recipes: Platform: ubuntu 18.04 Tags:
Conclusion
In this detailed article we learnt about the Chef Configuration Management tool with its basic understanding and overview of its components with installation and configuration settings. You can learn more about Chef by visiting the Chef website i.e. https://www.chef.io/