This solution provides in-depth guidance on how to manage Compute Engine
images. Images provide the base operating environment for applications that run
in Compute Engine, and they are critical to ensuring your application
deploys and scales quickly and reliably. You can also use images to archive
application versions for disaster recovery or rollback scenarios.
Understanding images
An
image
in Compute Engine is a cloud resource that provides a reference to an
immutable disk. That disk representation is then encapsulated using a few data
formats.
An image is a bundle of the raw bytes used to create a prepopulated hard disk.
Written on any formatted disk is a partition table that points to one or more
partitions that contain data. For an image to be bootable, it must contain the
following:
For a disk to be imported as a Compute Engine
image, the disk's bytes must be written to a file named
disk.raw
.
After the complete sequence of bytes from the disk are written to the file, the
file is archived using the
tar format
and then compressed using the GZIP format. You can then upload the resulting
*.tar.gz
file to Cloud Storage and register it as an image in
Compute Engine, as shown in the preceding diagram. After you register an
image, you can use it to create exact replicas of the original disk in any
region of Google Cloud. The newly registered images are often used as boot
volumes for Compute Engine instances.
For a more basic introduction to these Compute Engine terms, see
Virtual machine instances
and
Images
in the documentation.
Choosing a boot image
The first step in using Compute Engine is to choose the image you want
as the operating system for your virtual machine (VM) instance. You can use
public images supplied by Google Cloud
,
which are updated on a regular basis. Google Cloud provides a variety of
operating systems, including Debian, Ubuntu, and CentOS, for your use at no
extra cost. Some operating systems, such as Red Hat Enterprise Linux and
Microsoft Windows, are premium images, which incur additional fees for every
hour that the instances run.
For more information on a particular image, such as automatic update policies,
security patching, and support channels, see the
Operating system details
section of the product documentation.
For added security, you can also use the
Trusted image feature
to define
an organization policy that restricts images in specific public image projects
from being used in the creation of boot images.
You can use the Google Cloud public images to boot a
Compute Engine instance, after which you can customize the instance to
run your application.
One approach to configuring your instance is to use the
startup script
to run the commands that deploy your application as it boots. Keep in mind that
this script runs every time the instance boots, so you must make the script
idempotent to avoid ending up in an inconsistent or partially configured state.
If your instances are part of a managed instance group, you can use the
Instance Group Updater to restart or rebuild your instances, which reruns your
startup script. A common practice is to use the startup script to run a
configuration management tool such as Chef or Ansible.
Creating customized images
While configuring an instance's startup script is a viable way to provision your
infrastructure, a more efficient method is to create a new custom image with
your configuration incorporated into the public image. You can customize images
in several ways:
The process of creating a custom image is called
baking
.
Baking your images has the following benefits:
- Shorter time from boot to application readiness.
- Enhanced reliability for application deployments.
- Easier rollback to previous versions.
- Fewer dependencies on external services during application bootstrap.
- Scaling up creates instances that contain identical software versions.
Manual baking
You can create a simple custom image by creating a new VM instance from a
public image
,
configuring the instance with the applications and settings that you want, and
then creating a custom image from that instance. Use this method if you can
configure your images from scratch manually rather than using
automated baking
or
importing existing images
.
You can create a simple custom image using the following steps:
- Create an instance from a public image
.
- Connect to the instance
.
- Customize the instance for your needs.
- Stop the instance
.
- Create a custom image
from the boot disk of that instance. This process requires you to delete
the instance but keep the boot disk.
Automated baking
Manual baking is an easy way to start if you have a small number of images, but
large numbers of images become difficult to audit and manage.
Packer
is an open source tool for making image creation more reproducible, auditable,
configurable, and reliable. You can also use Packer as part of a Spinnaker
pipeline to produce images that are deployed to clusters of instances.
Importing existing images
You can import boot disk images from their existing infrastructure to
Compute Engine by using the
virtual disk import tool
, which
automates the image import process. For Linux machines, here's an
in-depth guide for manually migrating RAW disk images, Amazon Machine Images (AMI) and VirtualBox images
.
Another option for importing your existing images is to use
Migrate to Virtual Machines
.
Migrate to Virtual Machines is a tool chain and service that facilitates the migration of
machines from one platform to another with minimal downtime using continuous
block-level replication. You can migrate your machines to Compute Engine and then use manual baking to create images.
Encrypting images
All disks in Compute Engine are encrypted by default using
Google's encryption keys. Images built from disks are also encrypted.
Alternatively, you can provide
your own encryption keys when your disks are created
.
After you create the disk, you can create an encrypted image by providing your
encryption keys to the image create command. For more information about
encryption at rest and customer-supplied encryption keys, see
Encryption at rest
in the Google Cloud documentation.
Image lifecycle
After you set up an image-build pipeline, you can use images to reliably
launch instances of an application. While the pipeline can handle creating
images, you must also ensure that your deployment mechanisms use the latest
versions of the images. Finally, you need a process to curate images, so that
old and obsolete images aren't used inadvertently.
Image families
Image families help you manage images in your project by
grouping related images together, so that you can roll forward and
roll back between specific image versions. For more information, see
Image families best practices
.
Deprecating an image
As an administrator, you can also
roll back the image
that the image family points to by deprecating the image using the following
command:
gcloud compute images deprecate my-application-v3-20161011 --state DEPRECATED
You can choose from various states of deprecation:
State
|
Description
|
DEPRECATED
|
Images that are no longer the latest, but can still be launched by users. Users will see a warning at launch that they are no longer using the most recent image.
|
OBSOLETE
|
Images that should not be launched by users or automation. An attempt to create an instance from these images will fail. You can use this image state to archive images so their data is still available when mounted as a non-boot disk.
|
DELETED
|
Images that have already been deleted or are marked for deletion in the future. These cannot be launched, and you should delete them as soon as possible.
|
Enforcing lifecycle policies
You can mark images for deletion or obsolescence by using the
gcloud compute images deprecate
command. You can attach metadata to images to
mark them for future deletion by providing one of the
--delete-in
or
--delete-on
flags. To attach metadata to mark images for future obsolescence,
provide the
--obsolete-in
or
--obsolete-on
flags. You can incorporate this
command into an image-build process to enforce an image-lifecycle policy that
restricts the proliferation of stale and expired images in your project. For
example, at the end of your image-build pipeline, you could include an
additional check for images that need to be deprecated or deleted and then
perform those actions explicitly.
While deprecated and deleted images are no longer shown through the API and UI
by default, you can still see them by providing the
--show-deprecated
flag.
To completely delete the image and its data, you must send an explicit
delete command
for that image.
Sharing images between projects
Organizations often create multiple Google Cloud projects to partition
their resources, environments, and user access. Isolating resources into
projects allows for granular billing, security enforcement, and segregated
networking. Although most cloud resources do not need to span multiple projects,
images are good candidates for sharing across projects. By using a shared set of
images, you can follow a common process to deliver images with best practices
for security, authorization, package management, and operations pre-configured
for the rest of the organization.
You share images by assigning
IAM roles
to an organization's projects. The project that contains the images you want to
share with other projects, referred to in the preceding diagram as the
"Image Creation Project," must have the following IAM roles and policies
applied:
- Allow users of the "Image User Group" to create instances from these images
by granting them the
compute.imageUser
role.
- Allow the "Image Creation User" to create instances in this project by
granting them the
compute.instanceAdmin
role.
- Allow the "Image Creation User" to create images and disks in this project
by granting them the
compute.storageAdmin
role.
Projects that you want to be able to use the shared images must allow users with
the
compute.imageUser
role to create instances by assigning them the
compute.instanceAdmin
role.
For more detailed instructions on sharing images between projects, see
Sharing images across projects in the Compute Engine documentation.
What's next
- Review image
import options
- Work with boot images
- Work with custom images