Deploying Chef Server on Oracle Compute Cloud Service Instances

Before You Begin

Purpose

This tutorial shows you how to install a Chef server on an Oracle Compute Cloud Service instance or virtual machine. You will use the Chef server to automate deployment and configuration of different products on Oracle Compute Cloud Service instances.

Time to Complete

30 minutes

Background

Chef is a configuration management tool written in Ruby and Erlang. This tutorial uses Chef Server version 12 without any premium features like Chef Manage or Chef Reporting.

What Do You Need?

Subscription to Oracle Compute Cloud Service. See Getting Started with Oracle Cloud.

Oracle Linux 6.6.
Secure Shell (SSH) key with a bit size of 2048 or higher. See "Generating an SSH Key Pair" in the tutorial Creating Oracle Compute Cloud Service Instances Using an Orchestration.
Reserved fixed public IP address for each Oracle Compute Cloud Service instance. See Reserving a Public IP Address in Using Oracle Compute Cloud Service.
A minimum of two Oracle Compute Cloud Service instances, one to deploy the Chef server and another to bootstrap the Chef client. See the tutorial Creating Oracle Compute Cloud Service Instances Using an Orchestration. Set up both instances to boot from a persistent boot disk. See Creating a Bootable Storage Volume in Using Oracle Compute Cloud Service.
For the Chef server and the Chef client to communicate, the instances hosting them should be in the same security list (seclist). Otherwise, you need an access rule to enable communication over the appropriate protocol or port. See Managing Security Lists in Using Oracle Compute Cloud Service. If you're setting up a multi-node deployment, create instances as described in the tutorial Creating Oracle Compute Cloud Service Instances Using an Orchestration.
Each instance must be in a security list that permits inbound SSH connections from outside Oracle Cloud.
- Instances created using the Create Instance wizard in the web console are added to the /Compute-<account>/default/default security list by default. SSH connections to all instances in this security list are permitted by the auto-created security rule, {{/Compute-<account>/<user>/DefaultPublicSSHAccess}}. Remember to add an SSH public key while creating the instances.
- For instances created using an orchestration, you can specify the /Compute-<account>/default/default security list in the orchestration. If no security list is specified, instances are added to the /Compute-<account>/default/default security list by default.
Cookbooks and recipes to install the Chef server. You can access the cookbooks and recipes either from a GitHub repository or from Oracle Technology Network (OTN).

Performing Preinstallation Tasks

Complete the following preinstallation tasks:

Connect to the Oracle Compute Cloud Service instance (running the Chef server) by using a secure shell (SSH).
Install the git client on your Chef server instance.
Configure passwordless SSH connection between the Chef server and all instances managed by the server.

Connecting to the Oracle Compute Cloud Service Instance by Using SSH

You access the instance from your local machine by using the ssh command in a UNIX command shell or from Windows by using Putty.

Sign in to the My Services application at http://cloud.oracle.com/sign_in. On the My Services Dashboard, click Consoles and select Oracle Compute Cloud Service. On the Oracle Compute Cloud Service Console, click the name of the instance and note down the IP address that appears in the instance name section.
In a command shell, run the ssh utility:
```
$> ssh -i path_to_SSH_private_key opc@IP_address_of_instance
```
In the preceding command:
- path_to_SSH_private_key is the path to the SSH private key file that matches the public key used when your instance was created.
- IP_address_of_instance is the public IP address of the instance in n.n.n.n format.
- opc is the user account.
  Note: This step works only in Oracle-provided instances. You may have a different user provisioned in your custom instances.
  
  For example:
```
$> ssh -i keys/id_rsa opc@192.0.2.100
```
If this is the first time that you're connecting to the instance, confirm the RSA key fingerprint of the instance. In response to the prompts in the ssh utility, enter yes, and then enter the passphrase for the SSH key (if you created a passphrase).

Installing the `git` Client

After logging in to the Oracle Compute Cloud Service instance where you want to install the Chef server, install the git client.

sudo yum install -y git

Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package git.x86_64 0:1.7.1-3.el6_4.1 will be installed
--> Processing Dependency: perl-Git = 1.7.1-3.el6_4.1 for package: git-1.7.1-3.el6_4.1.x86_64
--> Processing Dependency: perl(Error) for package: git-1.7.1-3.el6_4.1.x86_64
--> Processing Dependency: perl(Git) for package: git-1.7.1-3.el6_4.1.x86_64
--> Running transaction check
---> Package perl-Error.noarch 1:0.17015-4.el6 will be installed
---> Package perl-Git.noarch 0:1.7.1-3.el6_4.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
==============================================================================================================================================================
 Package                            Arch                           Version                                    Repository                                 Size
==============================================================================================================================================================
Installing:
 git                                x86_64                         1.7.1-3.el6_4.1                            public_ol6_latest                         4.6 M
Installing for dependencies:
 perl-Error                         noarch                         1:0.17015-4.el6                            public_ol6_latest                          28 k
 perl-Git                           noarch                         1.7.1-3.el6_4.1                            public_ol6_latest                          27 k
Transaction Summary
==============================================================================================================================================================
Install       3 Package(s)
Total download size: 4.7 M
Installed size: 15 M
Downloading Packages:
(1/3): git-1.7.1-3.el6_4.1.x86_64.rpm                                                                                                  | 4.6 MB     00:05
(2/3): perl-Error-0.17015-4.el6.noarch.rpm                                                                                             |  28 kB     00:00
(3/3): perl-Git-1.7.1-3.el6_4.1.noarch.rpm                                                                                             |  27 kB     00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                         794 kB/s | 4.7 MB     00:06
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 1:perl-Error-0.17015-4.el6.noarch                                                                                                          1/3
  Installing : perl-Git-1.7.1-3.el6_4.1.noarch                                                                                                            2/3
  Installing : git-1.7.1-3.el6_4.1.x86_64                                                                                                                 3/3
  Verifying  : git-1.7.1-3.el6_4.1.x86_64                                                                                                                 1/3
  Verifying  : perl-Git-1.7.1-3.el6_4.1.noarch                                                                                                            2/3
  Verifying  : 1:perl-Error-0.17015-4.el6.noarch                                                                                                          3/3
Installed:
  git.x86_64 0:1.7.1-3.el6_4.1
Dependency Installed:
  perl-Error.noarch 1:0.17015-4.el6                                      perl-Git.noarch 0:1.7.1-3.el6_4.1
Complete!

Configuring Passwordless SSH Connection Between the Chef Server and Its Managed Instances

For the Chef server to connect to its managed client instances, you configure a passwordless SSH key connection between the server and the clients. That is, the instance hosting the Chef server must be able to open passwordless SSH connections to the instance hosting the Chef clients.

Before configuring a passwordless SSH connection, ensure that you've completed the "What Do You Need?" section in Before You Begin.

Step 1: Generate an SSH key pair on the instance that hosts the Chef server.

From your local workstation, using SSH, connect as opc to the instance that hosts the Chef server:
```
ssh opc@<chef_server_instance_public_ip> -i <opc_chef_server_private_key>
```
In the previous example, opc_chef_server_private_key is the private key of the public/private key pair that you generated when you created the Chef server instance.

Generate an SSH key pair but don’t specify a passphrase:

[opc@chefserver ~]$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/opc/.ssh/id_rsa): /home/opc/.ssh/chefkey
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/opc/.ssh/chefkey.
Your public key has been saved in /home/opc/.ssh/chefkey.pub.
The key fingerprint is:
fa:75:f5:61:48:24:f3:c9:f9:5c:76:07:42:8a:b3:29 opc@acme
The key's randomart image is:
+--[ RSA 2048]----+
|           +o..  |
|         . .*.o. |
|        o .  *  =|
|         +  . +.+|
|      E S    ..= |
|       o     ....|
|      .   . .   .|
|       . . .     |
|        .        |
+-----------------+

Note the full path and name of the files containing the public and private keys: /home/opc/.ssh/chefkey and /home/opc/.ssh/chefkey.pub.
Display the value of the public key that you just generated, and leave the SSH connection terminal open:
```
cat ~/.ssh/chefkey.pub
```
Your result should look similar to the following example:
```
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu143yaGM/ysaV1RSMPes9c5Tc3ML+lPLDJR+XGs8annPbmrs8XzlGHev47cs2V1p2sUsnyFj1AuNohymnyKMyeVq7vnZy1hkuPd7IXR0BXTjg/26vkLpmsiAb6ORq0kLGzHDW7j1Zl0ujYHnvnnqOz30IE3/rwyl285Xgxkm2wqnoTBOpb/kH4O4B61eBQX9bEC7C4vDiZ+hH0Lj/a6yWDVVqOwiOd0du+e4e5y08rJySP04lcMYBZyoazmkDVCJBElk6mr2Ge3QFYvExdKfcqmCIBue5S1l0v0LWRsK1oPemvjPmk/zw8CuihFpNPjiYzqpMus27a5AghEy/bF+Kw== opc@acme
```
Note: Alternatively, copy the public key value to an offline text file on your local workstation. Make sure that the key value that you copy doesn't include any extra characters and spaces. The entire key must be on one line.

Step 2. Append the public key (from Step 1) to the `/home/opc/.ssh/authorized_keys` file on each instance that hosts a Chef client.

From your local workstation, using SSH, connect as opc to the instances that hosts the Chef clients:
```
ssh opc@<chef_client_instance_public_ip> -i <opc_chef_client_private_key>
```
In the previous example, opc_chef_client_private_key is the private key of the public/private key pair that you generated creatingwhen you created the Chef client instance.
Open the /home/opc/.ssh/authorized_keys file in a text editor:
```
vi ~/.ssh/authorized_keys
```

Copy the public key value from Step 1 and append it as a new line after the existing keys in the file. Don’t include any extra characters and spaces. The entire key must be on one line.

Before: This sample file contains one key.

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA0ibAEmysI4o1zfb4dOJIyaN67pya8AnOozVewilpv560jiagTzwrPG8bmK4GL3KEUc3lxZ/Xhj7RvdOD0qMAx0fFB3r80ZSy1KlkIXwKumUY+YBMyn1xdMluWS/J4JWKBpuoOMNTGy7QdCPIHrt07OnwSxvZsoyTsh9QZ/eJv4qR0YaFkAHyH9Si2hTC/6G6CZdXkw93LyEtW1ykxxkSJB6JYwB8FsBMcXPvYJ5CiR30fKqo6GP+WTz1xbTbahLLO31mx/qSDntcXEYgfpw7Abi8W6LSkExFOxrsKir8QqZregznVeWPiht9kf4PT9C3WOoDzA0aF1q+g1CJ1EhZow== joe@acme

After: The file contains two keys, each on a separate line, and the new key is the last line.

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA0ibAEmysI4o1zfb4dOJIyaN67pya8AnOozVewilpv560jiagTzwrPG8bmK4GL3KEUc3lxZ/Xhj7RvdOD0qMAx0fFB3r80ZSy1KlkIXwKumUY+YBMyn1xdMluWS/J4JWKBpuoOMNTGy7QdCPIHrt07OnwSxvZsoyTsh9QZ/eJv4qR0YaFkAHyH9Si2hTC/6G6CZdXkw93LyEtW1ykxxkSJB6JYwB8FsBMcXPvYJ5CiR30fKqo6GP+WTz1xbTbahLLO31mx/qSDntcXEYgfpw7Abi8W6LSkExFOxrsKir8QqZregznVeWPiht9kf4PT9C3WOoDzA0aF1q+g1CJ1EhZow== joe@acme
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAu143yaGM/ysaV1RSMPes9c5Tc3ML+lPLDJR+XGs8annPbmrs8XzlGHev47cs2V1p2sUsnyFj1AuNohymnyKMyeVq7vnZy1hkuPd7IXR0BXTjg/26vkLpmsiAb6ORq0kLGzHDW7j1Zl0ujYHnvnnqOz30IE3/rwyl285Xgxkm2wqnoTBOpb/kH4O4B61eBQX9bEC7C4vDiZ+hH0Lj/a6yWDVVqOwiOd0du+e4e5y08rJySP04lcMYBZyoazmkDVCJBElk6mr2Ge3QFYvExdKfcqmCIBue5S1l0v0LWRsK1oPemvjPmk/zw8CuihFpNPjiYzqpMus27a5AghEy/bF+Kw== opc@acme

Save and close the file.
Repeatsteps a through d on each instance that hosts a Chef client.

Step 3. Test the SSH connection between the Chef server and a Chef client instance.

From your local workstation, using SSH, connect as opc to the instance that hosts the Chef server:
```
ssh opc@<chef_server_instance_public_ip> -i <opc_chef_server_private_key>
```
From the Chef server instance, using SSH, connect as opc to a Chef client instance.
```
ssh opc@<chef_client_instance_public_ip> -i ~/.ssh/chefkey
```

Setting Up the Chef Server

Using SSH, log on to the Oracle Compute Cloud Service instance where you want to install the Chef server and clone the compute-cloud-service-demos GitHub repository.
Clone the GitHub repository:
```
git clone https://github.com/oracle/compute-cloud-service-demos.git
```
The cloned repository contains a script that automates installation and configuration of the Chef server.

Alternatively, you can download the scripts that are required to set up the Chef server from the following link:

http://www.oracle.com/technetwork/topics/cloud/whatsnew/compute-cloud-service-2760092.html

Create a folder named compute-cloud-service-demos and unpack the scripts to the folder:
```
mkdir compute-cloud-service-demos | tar -xvzf master-compute-cloud-service-demos.tgz -C compute-cloud-service-demos
```
Open the /etc/profile file in an editor, and then search for the umask entry:
If the umask value isn't set to 022, or if the entry is missing, modify or add it, and then save and close the file:
```
umask 022
```
Install the Chef server:
```
./compute-cloud-service-demos/src/chef-server/oracle-ccs-deploy-chef.py --username your_username --email your_email --org your_organization --orgname your_organization_description
```
The script will prompt for the Chef Administrator user password.

In the previous example:
- your_username sets the Chef's administrator user name and defaults to admin.
- your_email sets the email address for the Chef administrator and defaults to no-replies@oracle.com.
- your_organization sets a short name for your Chef organization and defaults to ccs-demo.
- your_organization_description sets a long name for your Chef organization and default to Oracle Compute Cloud Services Demo.
Run the knife utility to verify your Chef server installation::
```
knife node list
```

Bootstrapping the Chef Clients

On your Chef server instance, bootstrap the instances that the Chef server will manage:

knife bootstrap instance.dns.name -x opc --sudo --bootstrap-version 12.3.0 -i ~/.ssh/chefkey

In the previous example, instance.dns.name is the domain name system (DNS) name of your second Oracle Compute Cloud Service instance that your Chef server will manage.

Doing old-style registration with the validation key at /home/opc/.chef/ccs-demo-validator.pem...
Delete your validation key in order to use your user credentials instead
Connecting to bb2150
bb2150 Installing Chef Client...
bb2150 --2015-06-05 20:02:58--  https://www.opscode.com/chef/install.sh
bb2150 Resolving www.opscode.com... 148.87.19.20
bb2150 Connecting to www.opscode.com|148.87.19.20|:80... connected.
bb2150 Proxy request sent, awaiting response... 200 OK
bb2150 Length: 18736 (18K) [application/x-sh]
bb2150 Saving to: ¿STDOUT¿
bb2150
100%[======================================>] 18,736      --.-K/s   in 0s
bb2150
bb2150 2015-06-05 20:02:59 (96.4 MB/s) - written to stdout [18736/18736]
bb2150
bb2150 Downloading Chef 12 for el...
bb2150 downloading https://www.opscode.com/chef/metadata?v=12&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64
bb2150   to file /tmp/install.sh.1367/metadata.txt
bb2150 trying wget...
bb2150 url  https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-12.3.0-1.el6.x86_64.rpm
bb2150 md5  c19fefcb3d033107e9fbdb3839312584
bb2150 sha256   4b7c846a9ad93564cc203a5ac99890431f7d6ad159c424aa89827fd772c9881d
bb2150 downloaded metadata file looks valid...
bb2150 downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-12.3.0-1.el6.x86_64.rpm
bb2150   to file /tmp/install.sh.1367/chef-12.3.0-1.el6.x86_64.rpm
bb2150 trying wget...
bb2150 Comparing checksum with sha256sum...
bb2150 Installing Chef 12
bb2150 installing with rpm...
bb2150 warning: /tmp/install.sh.1367/chef-12.3.0-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
bb2150 Preparing...                ########################################### [100%]
bb2150    1:chef                   ########################################### [100%]
bb2150 Thank you for installing Chef!
bb2150 Starting first Chef Client run...
bb2150 Starting Chef Client, version 12.3.0
bb2150 Creating a new client identity for <your Oracle compute cloud service instance endpoint> using the validator key.
bb2150 resolving cookbooks for run list: []
bb2150 Synchronizing Cookbooks:
bb2150 Compiling Cookbooks...
bb2150 [2015-06-05T20:03:50-04:00] WARN: Node <your Oracle compute cloud service instance endpoint> has an empty run list.
bb2150 Converging 0 resources
bb2150
bb2150 Running handlers:
bb2150 Running handlers complete
bb2150 Chef Client finished, 0/0 resources updated in 4.719307443 seconds

Uploading Cookbooks, Roles, and Environments

The cloned GitHub repository contains sample cookbooks, roles, and environments that you use to deploy different software stacks on Oracle Compute Cloud Service. If you downloaded the scripts from Oracle Technology Network, you’ll find these cookbooks, roles, and environments in the src folder of the zip file that you downloaded and extracted to your local directory on the Oracle Compute Cloud Service instance.

On your Chef server instance, upload the cookbooks, roles, and environments:

cp -rp compute-cloud-service-demos/src/chef-repo .
knife cookbook upload -a
knife upload /roles
knife upload /environments

Updating Nodes Permissions

Before you can run a Chef client on your managed instance, you update the nodes permissions so that a Chef client can update the Chef server after a successful run.

On your Chef server instance, update the nodes permissions:

knife download /acls/nodes
cd /home/opc/chef-repo/acls/nodes
sed -i '31i "clients",' *.json
knife upload /acls/nodes

Note: If you update permissions for another client or node, update the sed command to be applicable just for the new node:

sed -i '31i "clients",' new_client_fully_qualified_name.json

Assigning Roles to a Node

On your Chef server instance, assign one or more roles to your Chef server instance:

knife node run_list add instance.dns.name 'role[name1],role[name2]'

In the previous example:

instance.dns.name is the DNS name of your second Oracle Compute Cloud Service instance that will be managed by your Chef server.
name1, name2 are the names of the roles that you want to assign to your node.

Note:

Running the Chef Client

After you upload a sample role with a sample cookbook to your Chef server, and after you assign the role to a bootstrapped instance, you update your managed instance.

On your Chef server instance, run the Chef client:

knife ssh -x opc 'role:base-role' 'sudo chef-client' –i ~/.ssh/chefkey

Starting Chef Client, version 12.3.0
<your Oracle compute cloud service instance endpoint> resolving cookbooks for run list: ["custom-ssh-banner"]
<your Oracle compute cloud service instance endpoint> Synchronizing Cookbooks:
<your Oracle compute cloud service instance endpoint>   - custom-ssh-banner
<your Oracle compute cloud service instance endpoint> Compiling Cookbooks...
<your Oracle compute cloud service instance endpoint> Converging 3 resources
<your Oracle compute cloud service instance endpoint> Recipe: custom-ssh-banner::default
<your Oracle compute cloud service instance endpoint>   * log[Starting custom-ssh-banner recipe...] action write
<your Oracle compute cloud service instance endpoint>
<your Oracle compute cloud service instance endpoint>   * file[/etc/custom-ssh-banner] action create
<your Oracle compute cloud service instance endpoint>     - create new file /etc/custom-ssh-banner
<your Oracle compute cloud service instance endpoint>     - update content in file /etc/custom-ssh-banner from none to d9343d
<your Oracle compute cloud service instance endpoint>     --- /etc/custom-ssh-banner    2015-06-05 20:06:24.550921707 -0400
<your Oracle compute cloud service instance endpoint>     +++ /etc/.custom-ssh-banner20150605-1627-1qmpen6  2015-06-05 20:06:24.550921707 -0400
<your Oracle compute cloud service instance endpoint>     @@ -1 +1,2 @@
<your Oracle compute cloud service instance endpoint>     +This banner was brought to you by Chef using custom-ssh-banner.
<your Oracle compute cloud service instance endpoint>   * file[/etc/profile.d/custom-ssh-banner.sh] action create
<your Oracle compute cloud service instance endpoint>     - create new file /etc/profile.d/custom-ssh-banner.sh
<your Oracle compute cloud service instance endpoint>     - update content in file /etc/profile.d/custom-ssh-banner.sh from none to cfc0cc
<your Oracle compute cloud service instance endpoint>     --- /etc/profile.d/custom-ssh-banner.sh   2015-06-05 20:06:24.603921706 -0400
<your Oracle compute cloud service instance endpoint>     +++ /etc/profile.d/.custom-ssh-banner.sh20150605-1627-1isborn 2015-06-05 20:06:24.603921706 -0400
<your Oracle compute cloud service instance endpoint>     @@ -1 +1,2 @@
<your Oracle compute cloud service instance endpoint>     +cat /etc/custom-ssh-banner; echo
<your Oracle compute cloud service instance endpoint>     - change mode from '' to '0755'
<your Oracle compute cloud service instance endpoint>
<your Oracle compute cloud service instance endpoint> Running handlers:
<your Oracle compute cloud service instance endpoint> Running handlers complete
<your Oracle compute cloud service instance endpoint> Chef Client finished, 3/3 resources updated in 3.044077698 seconds

Verify the Chef client run:

knife status 'role:base-role'

0 minutes ago, <your Oracle compute cloud service instance endpoint>, oracle 6.6.

Want to Learn More?

Credits

Lead Curriculum Developer: Anirban Ghosh
Other Contributors: Eder Zechim, Michael Yang, S. Matt Taylor Jr., Irina Mok