Chef

Table Of Contents

knife google

Google Compute Engine is a cloud hosting platform that offers scalable and flexible virtual machine computing. The knife google subcommand is used to manage API-driven cloud servers that are hosted by Google Compute Engine.

Note

Review the list of common options available to this (and all) Knife subcommands and plugins.

Install this plugin

To install the knife google plugin using RubyGems, run the following command:

$ /opt/chef/embedded/bin/gem install knife-google

where /opt/chef/embedded/bin/ is the path to the location where the chef-client expects Knife plugins to be located. If the chef-client was installed using RubyGems, omit the path in the previous example.

Bootstrap and SSH

Before a bootstrap operation can be run on a node, SSH keys must be set up correctly. In Google Compute Engine SSH keys are stored in project metadata that is copied over to new servers and placed in a user’s ~/.ssh/authorized_keys file.

Note

If the SSH keys are not already set up, they can be created using the ssh-keygen program.

Add the SSH key using the Metadata page in the Google Compute Engine console. Paste the SSH key in the user’s ~/.ssh/id_rsa.pub file, ensuring to prefix the SSH key with the same user name that would be specified when using the --ssh-user option used by the knife server create command.

For example:

user_name:ssh-rsa AYAAB3Nwejwejjfjawlwl990sefjsfC5lPulcP4eZB+z1zcMF
76gTV4vojT/SWXymTfGpBL2KHTmF4jnGfEKPwjHIiLrZNHM2ISMi/atlKjOoUCVT
AvUyjqqp3z2KVXSP9P50Kgf8JYWjjXKApiZHkJOHJZ8GGf7aTnRU9NEGLbQK6Q1k
4UHbVG4ps4kSLWsJ7eVcu981GvlwP3ooiJ6YWcOX9PS58d4SNtq41/XaoLibKt/Y
Wzd/4tjYwMRVcxJdAy1T2474vkU/Qr7ibFinKeJymgouoQpEGhF64cF2pncCcmR7
zRk7CzL3mhcma8Zvwj234-2f3/+234/AR#@R#y1EEFsbzGbxOJfEVSTgJfvY7KYp
329df/2348sd3ARTx99 mymail@myhost

where user_name: is the prefix added to the SSH key.

disk create

The disk create argument is used to create a disk hosted by Google Compute Engine. The name of the disk, its size, and the zone in which it will be created must be specified.

Syntax

This argument has the following syntax:

$ knife google disk create (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-s SIZE, --gce-disk-size SIZE
The size of a disk, in GB.
-Z ZONE, --gce-zone ZONE
The Google Compute Engine zone in which a disk is located.

disk delete

The disk delete argument is used to delete a disk hosted by Google Compute Engine. If the disk is currently attached to a running server, it will not be deleted.

Syntax

This argument has the following syntax:

$ knife google disk delete (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-Z ZONE, --gce-zone ZONE
The Google Compute Engine zone in which a disk is located.

disk list

The disk list argument is used to view a list of disks that are hosted by Google Compute Engine.

Syntax

This argument has the following syntax:

$ knife google disk list (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-Z ZONE, --gce-zone ZONE
The Google Compute Engine zone in which a disk is located.

Examples

View a list of disks

To view a list of disks, enter:

$ knife google disk list

to return something similar to the following:

Name                Zone           Source Snapshot  Size (In GB)  Status
jay-scratch         us-central2-a                   10            ready
pd-fuse             us-central2-a                   10            ready
pd28g               us-central2-a                   28            ready

project list

The projects list argument is used to view a list of projects that are hosted by Google Compute Engine.

Syntax

This argument has the following syntax:

$ knife google project list (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-L, --with-limits
Use to set a quota limit.

Examples

None.

region list

The region list argument is used to view a list of regions that are hosted by Google Compute Engine.

Syntax

This argument has the following syntax:

$ knife google region list (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-L, --with-limits
Use to set a quota limit.

Examples

None.

server create

The server create argument is used to create a new Google Compute Engine cloud instance. This will provision a new image in Google Compute Engine, perform a chef-client bootstrap (using the SSH protocol), and then install the chef-client on the target system so that it can be used to configure the node and to communicate with a Chef server.

Syntax

This argument has the following syntax:

$ knife google server create SERVER_NAME [RUN_LIST] (options)

Options

This argument has the following options:

--bootstrap-version VERSION
The version of the chef-client to install.
-d DISTRO, --distro DISTRO
The template file to be used during a bootstrap operation. The following distributions are supported: chef-full (the default bootstrap), centos5-gems, fedora13-gems, ubuntu10.04-gems, ubuntu10.04-apt, ubuntu12.04-gems, and the name of a custom bootstrap template file. When this option is used, Knife will search for the template file in the following order: the bootstrap/ folder in the current working directory, the bootstrap/ folder in the chef-repo, the bootstrap/ folder in the ~/.chef/ directory, or a default bootstrap file. Do not use the --template-file option when --distro is specified.
-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
--gce-boot-disk-name DISK
The name of the persistent boot disk.
--gce-boot-disk-size SIZE
The size (in GB) of the persistent boot disk. This must be a value between 10 and 10000. Default value: 10.
--gce-image-project-id IMAGE_PROJECT_ID
The ID for the project that contains the image.
--gce-metadata Key=Value[,Key=Value...]
A list of metadata key-value pairs that are associated with an instance.
--gce-public-ip IP_ADDRESS
The public IP address for Google Compute Engine. Options: EPHEMERAL, a static IP address, or NONE. Default value: EPHEMERAL.
--gce-server-connect-ip PUBLIC
Indicates whether the IP address for Google Compute Engine is PUBLIC or PRIVATE. Default value: PUBLIC.
--gce-service-account-name NAME
The name of the service account associated with the server. The typical format is similar to: 12345678@project.gceserviceaccount.com. Default value: 'default'.
--gce-service-account-scopes SCOPE1, SCOPE2, SCOPE3
Additional metadata for the server.
--hint HINT_NAME[=HINT_FILE]
An Ohai hint to be set on the target of the bootstrap. The hint is contained in a file and is formatted as JSON: {"attribute":"value","attribute":"value"...}. HINT_NAME is the name of the hint and HINT_FILE is the name of the hint file located at /etc/chef/ohai/hints/HINT_FILE.json. Use multiple --hint options in the command to specify multiple hints.
--[no-]host-key-verify
Use --no-host-key-verify to disable host key verification. Default setting: --host-key-verify.
-i IDENTITY_FILE, --identity-file IDENTITY_FILE
The SSH identity file used for authentication. Key-based authentication is recommended.
-I IMAGE, --gce-image IMAGE
The name of the virtual application template or image that will be used to create the virtual machine. Default: gcompute8-standard.
-j JSON_ATTRIBS, --json-attributes JSON_ATTRIBS
A JSON string that is added to the first run of a chef-client.
-m MACHINE_TYPE, --gce-machine MACHINE_TYPE
The machine type for the instance running in Google Compute Engine server. For example, n1-highcpu-2 or n1-highcpu-2-d. The full list of available machine types: https://developers.google.com/compute/docs/instances.
-n NETWORK_NAME, --gce-network NETWORK_NAME
The name of the network in which an instance is running.
-N NODE_NAME, --node-name NODE_NAME
The name of the node.
--[no-]gce-auto-server-restart
Use to specify if Google Compute Engine will automatically restart the virtual machine instance if it is terminated by non-user initiated actions. Default value: true.
--[no-]gce-auto-server-migrate
Use to specify if Google Compute Engine can migrate the virtual machine instance without downtime, prior to periodic infrastructure maintenance. Default value: true.
-p PORT, --ssh-port PORT
The SSH port. Default value: 22.
-P PASSWORD, --ssh-password PASSWORD
The SSH password. This can be used to pass the password directly on the command line. If this option is not specified (and a password is required) Knife will prompt for the password.
--prerelease
Indicates that pre-release gems should be installed.
-r RUN_LIST, --run-list RUN_LIST
A comma-separated list of roles and/or recipes to be applied.
--secret SECRET
The encryption key that is used for values contained within a data bag item.
--secret-file FILE
The path to the file that contains the encryption key.
-T TAG1,TAG2,TAG3, --gce-tags TAG1,TAG2,TAG3
A list of tags that are associated with an instance.
--template-file TEMPLATE
The path to a template file that will be used during a bootstrap operation. Do not use the --distro option when --template-file is specified.
-u USER_DATA_FILE, --user-data USER_DATA_FILE
The user data file used to provision Google Compute Engine.
-w GATEWAY, --ssh-gateway GATEWAY
The SSH tunnel or gateway that is used to run a bootstrap action on a machine that is not accessible from the workstation.
-x USER_NAME, --ssh-user USER_NAME
The SSH user name.
-Z ZONE, --gce-zone ZONE
The Google Compute Engine zone in which a disk is located.

Examples

Create an instance

To create a server:

$ knife google server create www1 -m n1-standard-1
    -I centos-6-v20130325 -Z us-central2-a -i ~/.ssh/id_rsa -x jdoe

Create a Jenkins continuous integration pipeline

The knife google server create subcommand can be used to deploy infrastructure using Chef recipes to Google Compute Engine. For example, creating a master node running Jenkins, hosted in Google Compute Engine:

knife google server create jenkins1 -Z us-central1-a -m n1-highcpu-2 -I debian-7-wheezy-v20131120 -r 'jenkins::master'

This command takes the following actions:

  • Creates a Debian virtual machine instance in the us-central1-a zone with machine type n1-highcpu-2
  • Registers it as a node named jenkins1 with the Chef server
  • Configures the run_list attribute on the node as jenkins::master
  • Uses the SSH protocol to run the chef-client with the jenkins::master recipe from the jenkins community cookbook

At the end of this process, a message similar to:

Chef Client finished, 19/21 resources updated in 40.207903203 seconds

will confirm that Jenkins has been installed successfully as a Jenkins master. This (and similar) Knife commands may be integrated into automation that can also spin up Jenkins testing systems to build a complete continuous integration pipeline backed by Google Compute Engine.

You can then use other Chef server features—like search—to manage the Jenkins pipeline as long as it is needed. When finished, just destroy part or all of it:

knife google server delete tester1 -y --purge

server delete

The server delete argument is used to delete one or more instances that are running in the Google Compute Engine cloud. To find a specific cloud instance, use the knife google server list argument. Use the knife node delete and knife client delete sub-commands to delete associated node and client objects (if required).

Syntax

This argument has the following syntax:

$ knife google server delete SERVER_NAME (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-N NAME, --node-name NAME
The name of the node to be deleted, if different from the server name. This must be used with the --purge option.
-p, --purge
Indicates that all corresponding nodes and clients on the Chef server will be destroyed, in addition to the Google Compute Engine node itself. This action (by itself) assumes that the node and client have the same name as the server; if they do not have the same names, then the --node-name option must be used to specify the name of the node.
-Z ZONE, --gce-zone ZONE
The Google Compute Engine zone in which a disk is located.

Examples

Delete every chef-client and node

To delete a server using the --purge option. which will delete every chef-client and node:

$ knife google server delete www1 --purge -Z us-central2-a

server list

The server list argument is used to find instances that are associated with a Google Compute Engine zone. The results may show instances that are not currently managed by the Chef server.

Syntax

This argument has the following syntax:

$ knife google server list (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)
-Z ZONE, --gce-zone ZONE
The Google Compute Engine zone in which a disk is located.

Examples

List specific instances

To view a list of instances associated with a project named “dev-01”, enter:

$ knife google server list -p dev-01

List all servers

To view a list of all servers in the us-central2-a zone, including those that may not be managed by the chef-client, enter:

$ knife google server list -Z us-central2-a

to return something similar to the following:

Name              Type           Image                 Public IP        Private IP      Disks               Zone           Status
chef-svr          n1-standard-1  gcel-12-04-v20130325  103.59.80.113    10.240.45.78                        us-central2-a  running
chef-workstation  n1-standard-1  gcel-12-04-v20130325  103.59.85.188    10.240.9.140                        us-central2-a  running
fuse-dev          n1-standard-1  gcel-12-04-v20130225  103.59.80.147    10.240.166.18   pd-fuse             us-central2-a  running
magfs-c1          n1-standard-2  gcel-12-04-v20130225  103.59.87.217    10.240.61.92                        us-central2-a  running
magfs-c2          n1-standard-2  gcel-12-04-v20130225  103.59.80.161    10.240.175.240                      us-central2-a  running
magfs-c3          n1-standard-2  gcel-12-04-v20130325  178.255.120.69   10.240.34.197   jay-scratch         us-central2-a  running
magfs-svr         n1-standard-4  gcel-12-04-v20130225  103.59.80.178    10.240.81.25    pd28g               us-central2-a  running

setup

The setup argument is used to set up authorization for a Google Compute Engine account. If the -f parameter is used to override the default credential file, it will also need to be used with all subcommands. Be sure to specify the project identifier (and not its name or number) to prevent 404 errors (even if the knife google setup command runs successfully).

Create a project in Google Cloud Platform, enable Google Compute Engine, and then set up the client identifier. Run the following command:

$ knife google setup

and then provide the project identifier (not the project name or number), the client identifier, the client secret, and authorization tokens when prompted. Open the URL in a browser when promoted. Ensure that the user account associated with the project and client identifier is used when authenticating.

By default, the credential and token information is stored in ~/.google-compute.json. Use the f <credential_file> flag to override this location.

Syntax

This argument has the following syntax:

$ knife google setup (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)

zone list

The zone list argument is used to get the list of available Google Compute Engine zones, including current status, number of deployed servers (and disks), and upcoming maintenance windows.

Syntax

This argument has the following syntax:

$ knife google zone list (options)

Options

This argument has the following options:

-f CREDENTIAL_FILE, --gce-credential-file CREDENTIAL_FILE
The credential file used by Google Compute Engine. (This file is created when running the knife google setup command.)

Examples

View a list of zones

To see a list of all zones, including status and maintenance windows, enter:

$ knife google zone list

to return something similar to the following:

Name            Status  Servers  Disks  Maintainance Window
europe-west1-a  up      0        0      2013-08-03 19:00:00 +0000 to 2013-08-18 19:00:00 +0000
europe-west1-b  up      0        0      2013-05-11 19:00:00 +0000 to 2013-05-26 19:00:00 +0000
us-central1-a   up      0        1      2013-08-17 19:00:00 +0000 to 2013-09-01 19:00:00 +0000
us-central1-b   up      0        0      2013-06-08 19:00:00 +0000 to 2013-06-23 19:00:00 +0000
us-central2-a   up      10       6      2013-05-25 19:00:00 +0000 to 2013-06-09 19:00:00 +0000