This guide explains how to ensure that the Monitoring agent , which you install
on your virtual machine (VM) instance, is authorized to send telemetry data to
Monitoring.
Authorization overview
Authorization
refers to the process of determining what permissions an
authenticated client has for a set of resources. Google Cloud authorizes
the Monitoring agent on a Compute Engine VM instance by using
application default credentials
(ADC).
The Monitoring agent supports ADC
that authenticate either a VM's attached service account,
or a service account key.
- An
attached service account
refers to a service account that's specific
to a
given resource, such as a VM. The service account has its own unique
credentials. ADC uses the VM's
metadata server
to obtain credentials for a service.
- A
service account key
refers to a private key used to
authorize the
key pair on a service account in a project, which lets you create an access
token. You use the token to provide an identity so that you can interact
with Google Cloud APIs on behalf of the service account.
We recommend that you configure the ADC
to authenticate an attached service account whenever possible, as the private
key requires local storage,
and that storage can be compromised
.
For more information about service account keys, see
Best practices for managing service account keys
.
For AWS EC2 VM instances, the Monitoring agent supports authentication only
using the service account key method.
Before you begin
Read this guide if any of the following applies to you:
If you're running very old Compute Engine instances or if you have
modified the access scopes or service account settings for your
Compute Engine instances, then you must complete the steps in this
guide before installing the agent. These VMs might not have the required
service account key file. For information about how to verify the access
scopes and service account settings of your instances, see
Verify Compute Engine credentials
.
On newly created Compute Engine VM instances, the access scopes and
service account settings are sufficient to run the agents.
If you're running Amazon EC2 (AWS VC2) VM instances, then do the following
before installing the agent:
Identify the
AWS hosting project
for the AWS account with your EC2
instances. The hosting project is a Google Cloud project whose sole purpose
is to store the metrics and logs from the EC2 instances in an AWS account.
If you have an
AWS connector project
for an AWS account,
then the connector project is the AWS hosting project.
If you don't have an AWS connector project for the account, then
search your existing Google Cloud projects to determine whether you
previously created an AWS hosting project for the account. If a
hosting project is found, then use it.
If you can't find an AWS hosting project for the account, then we
recommend that you create a Google Cloud project to serve as the
hosting project. If you have EC2 instances in multiple AWS accounts,
then create a Google Cloud project for each account. We also recommend that
you use a naming convention for these projects or that you use
project labels
,
so that you can identify your hosting projects.
We don't recommend that you re-use an existing
Google Cloud project, especially if it contains any Google Cloud resources.
Complete the steps to authorize the agent by using a
service account key file.
To verify your access scopes, do the following:
- Query the access scopes by running the following command on
your Compute Engine instance:
curl --silent --connect-timeout 1 -f -H "Metadata-Flavor: Google" http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes
In the command output, if the access scope
https://www.googleapis.com/auth/cloud-platform
is listed, then you have sufficient authorization.
If
https://www.googleapis.com/auth/cloud-platform
isn't
listed, then you require two access scopes, one from each of the
following "logging" and "monitoring" pairs:
https://www.googleapis.com/auth/logging.write
or
https://www.googleapis.com/auth/logging.admin
https://www.googleapis.com/auth/monitoring.write
or
https://www.googleapis.com/auth/monitoring.admin
To modify your access scopes, do the following:
-
In the Google Cloud console, go to the
VM instances
page:
Go to
VM instances
If you use the search bar to find this page, then select the result whose subheading is
Compute Engine
.
- If necessary, click the drop-down list of Google Cloud projects
and select the name of your project.
- Select
VM instances
from the navigation menu, select the
Instances
tab, and the select the name of your VM.
- Shut down the VM by clicking
stop
Stop
.
- After the VM stops, click
edit
Edit
.
- Locate the
Access scopes
in the
Identity and API access
section of the page, and then select
Set access for each API
.
- For
Stackdriver Logging API
and
Stackdriver Monitoring API
entries,
select the
Write Only
.
- Click
Save
, and then restart the VM by clicking
play_arrow
Start/Resume
.
Use a service account
Authentication
refers to the process of determining a client's identity.
For authentication, we recommend using a
service account
, a special kind
of account that is typically used by an application or a workload,
rather than a person. For more information, see
Service accounts overview
.
You can use service accounts for authentication regardless of where your code
runs: on Compute Engine, App Engine, or on-premise.
For more information, see
Authentication at Google
.
This section describes how to create a new service account and grant it
the necessary roles, and how to update an existing service account if it
doesn't have the necessary roles.
Create a service account
To create a service account, complete the
Creating a service account
procedures with the following information:
Select the Google Cloud project in which to create the service account.
For Compute Engine instances, choose the project in which you
created the instance.
For Amazon EC2 instances, select the
AWS hosting project
.
In the
Role
drop-down menu, select the following roles:
- Monitoring
>
Monitoring Metric Writer
. This authorizes the
Monitoring agent.
If you will also install the Logging agent, then add the
following role for that agent:
- Logging
>
Logs Writer
. This authorizes the Logging
agent.
If you plan to authenticate using a service account key,
then select
JSON
as the
Key type
and click
Create
.
When you click
Create
, a file that contains a service account key is
downloaded to your local system. For more information, see
Create and delete service account keys
.
Next, configure your service account and settings based on whether you authorize
by using attached service accounts or by using service account private keys.
Verify and modify roles of an existing service account
You can use the Google Cloud console to determine which roles an existing service
account has, and to add any necessary roles that are missing:
-
In the Google Cloud console, go to the
IAM
page:
Go to
IAM
If you use the search bar to find this page, then select the result whose subheading is
IAM & Admin
.
If necessary, click the drop-down list of Google Cloud projects
and select the name of your project.
If you don't see a list of IAM principals
(users and service accounts), then select the
Permissions
tab.
In the
View by Principals
list, locate the entry for the
service account. The
Role
column lists the roles granted to
the service account.
If your service account does not have the necessary roles for the
Monitoring agent , then use the following steps to add the roles described in
Create a service account
:
- Click
edit
Edit
in the entry for the service account.
- Click
Add Another Role
to add any missing roles.
- Click
Save
.
Authorize with an attached service account
To authorize the Monitoring agent installed on a Compute Engine VM instance
that has an attached service account, do the following:
Ensure that you
verified the access scopes of your VM
.
Grant your service account the
least privileged
IAM roles possible. For the required roles, see the
Create a service account
section of this page.
Attach the service account
to the VM where the agent is running.
If you haven't already installed the agent, then install it.
For information about how to install the agent, see
Installing the agent
.
Authorize with a service account key
To authorize the Monitoring agent installed on a VM instance by using
service account private keys, do the following:
Transfer the service account key file from your local system to your VM instance:
Create an environment variable to point to the service account key file
on your
local system. The following example creates a variable called
CREDS
:
CREDS=~/Downloads/
PROJECT-NAME
-
KEY-ID
.json
Complete the steps shown in the following table:
Compute Engine
-
In the Google Cloud console, go to the
VM instances
page:
Go to
VM instances
If you use the search bar to find this page, then select the result whose subheading is
Compute Engine
.
Identify the
INSTANCE_NAME
and
INSTANCE_ZONE
for your VM.
On your local system, run a
Google Cloud CLI
command to copy the key file from your local system to your VM instance:
REMOTE_USER="$USER"
INSTANCE="
INSTANCE_NAME
"
ZONE="
INSTANCE_ZONE
"
gcloud compute scp "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json" --zone "$ZONE"
On your Compute Engine instance, move the service account key file
from the temporary location to a permanent location,
and for Linux, ensure that the service account key file is readable only
by
root
.
These are the locations where the agent expects your key file:
- Linux VMs
:
/etc/google/auth/application_default_credentials.json
- Linux VMs:
Any location you store in the environment
variable
GOOGLE_APPLICATION_CREDENTIALS
, which must be visible to
the agent's process. For configuration information, see
Set
GOOGLE_APPLICATION_CREDENTIALS
.
For example, on Linux you can run the following script which moves the
service account key file to the default location, and then sets the appropriate
permissions:
CREDENTIALS_FILE_LOCATION="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$CREDENTIALS_FILE_LOCATION"
sudo chown root:root "$CREDENTIALS_FILE_LOCATION"
sudo chmod 0400 "$CREDENTIALS_FILE_LOCATION"
Amazon EC2
For your VM, identify
YOUR-INSTANCE'S-ID
and
YOUR-INSTANCE'S-AWS-REGION
.
On your local system, use
scp
to copy the service account key file from your local system to your VM
instance:
KEY="
YOUR-SSH-KEY-PAIR-FILE
"
INSTANCE="ec2-
YOUR-INSTANCE'S-ID
.
YOUR-INSTANCE'S-AWS-REGION
.compute.amazonaws.com"
# The remote user depends on the installed OS: ec2-user, ubuntu, root, etc.
REMOTE_USER="
EC2-USER
"
scp -i "$KEY" "$CREDS" "$REMOTE_USER@$INSTANCE:~/temp.json"
The value of the
INSTANCE
environment variable assumes that your
EC2 instance isn't in the
us-east-1
region. To learn how to define the
INSTANCE
variable when your instance is in the
us-east-1
region, see
naming convention defined by Amazon
.
On your EC2 instance, move the service account key file from the
the temporary location to one of the following locations, and for Linux,
ensure that the service account key file is readable only by
root
.
These are the locations where the agent expects your key file:
Linux VMs
:
/etc/google/auth/application_default_credentials.json
Windows VMs
:
C:\ProgramData\Google\Auth\application_default_credentials.json
Linux and Windows VMs: Any location you store in the environment
variable
GOOGLE_APPLICATION_CREDENTIALS
, which must be visible to
the agent's process. For configuration information, see
Set
GOOGLE_APPLICATION_CREDENTIALS
.
For example, on Linux you can run the following script which moves the
service account key file to the default location, and then sets the appropriate
permissions:
CREDENTIALS_FILE_LOCATION="/etc/google/auth/application_default_credentials.json"
sudo mkdir -p /etc/google/auth
sudo mv "$HOME/temp.json" "$CREDENTIALS_FILE_LOCATION"
sudo chown root:root "$CREDENTIALS_FILE_LOCATION"
sudo chmod 0400 "$CREDENTIALS_FILE_LOCATION"
Your VM instance now has the service account key file that the agent needs.
Next, install or restart the agent:
Set
GOOGLE_APPLICATION_CREDENTIALS
This section shows how to set the environment variable
GOOGLE_APPLICATION_CREDENTIALS
so that it is visible to the agent's process.
Linux
Edit the following configuration file, or create the file if it doesn't
exist:
/etc/default/stackdriver-agent
Add the following to the configuration file:
GOOGLE_APPLICATION_CREDENTIALS=
PATH_TO_CREDENTIAL_FILE
Restart the agent by running the following command on your VM instance:
sudo service stackdriver-agent restart
Windows
In PowerShell, run the following commands as administrator to set the
GOOGLE_APPLICATION_CREDENTIALS
system environment variable for the
Ops Agent to use:
[Environment]::SetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS", "
PATH_TO_CREDENTIAL_FILE
", "Machine")
Restart the agent by running the following command on your VM instance:
Restart-Service -Name StackdriverMonitoring