This page is an in-depth continuation of the discussion and examples
presented in
Metrics, time series, and resources
and
Components of the metric model
. To get the
most from this page, read those pages first.
This page is intended for readers who want a deeper understanding of time
series and the implementation of the Cloud Monitoring metric
model, particularly those readers who are using the Monitoring API.
This includes users of custom (user-defined) metrics.
This page maps the general metric model to the Cloud Monitoring API, so it
is useful for anyone using the Monitoring API directly. Users whose
primary interaction is with the Google Cloud console probably don't need this
level of detail.
It is also useful to anyone using a third-party integration like
OpenTelemetry
. As data from an external source
reaches Cloud Monitoring through an integration, it is mapped to the
Cloud Monitoring metric model and treated like any other metric data.
For more on collecting your own metrics, see
User-defined metrics overview
.
Review of the metric model
A prerequisite for understanding the structure of time series is familiarity
with the metric model used by Cloud Monitoring. This model is described in
Metrics, time series, and resources
. This section
provides a brief review.
In general terms, monitoring data is recorded in
time series
. Each individual
time series includes three pieces of information relevant to this discussion:
Each time series holds the measurements from a single physical or logical source.
If your environment has hundreds of resources, each contributes at least one
time series. In fact, if your metric type and monitored-resource type have
labels, you get one time series per combination of label values; see
Cardinality
for more information.
A sample time series
A time series is represented as an instance of a
TimeSeries
object.
The following is a complete instance of a single time series:
{
"metric"
: {
"labels": {
"log": "kubelet",
"severity": "DEFAULT"
},
"type": "logging.googleapis.com/log_entry_count"
},
"resource"
: {
"type": "gce_instance",
"labels": {
"instance_id": "5106847938295940291",
"zone": "us-central1-a",
"project_id": "a-gcp-project"
}
},
"metricKind": "DELTA",
"valueType": "INT64",
"points"
: [
{
"interval": {
"startTime": "2019-12-20T20:25:38Z",
"endTime": "2019-12-20T20:26:38Z"
},
"value": {
"int64Value": "20"
}
}
]
}
Most time series include a lot more data points; this one covers
a one-minute interval. All time series have the same structure, with the
following fields:
The
metric
field records
- The metric-label values for this particular time series, representing one
combination of label values.
- The metric type with which the data is associated. The metric type specifies
the available labels and describes what is represented by the
data points.
The information in this field is described in more detail in
Metrics
.
The
resource
field records:
- The resource-label values for this particular time series, representing one
combination of label values.
- The specific monitored resource from which the data was collected.
The information in this field is described in more detail in
Monitored-resource objects
.
The
metricKind
and
valueType
fields tell you how to interpret the values.
For more information, see
Metric kinds and types
.
The
points
field is an array of timestamped values. The metric type tells
you what the values represent. The sample time series has an array with a
single data point; in most time series, the array has many more values.
For a live example that retrieves time series data, see
Time series: data from
a monitored resource
.
This rest of this page looks at the information in the
metric
and
resource
fields
in more detail.
Monitored resources
Cloud Monitoring collects data from monitored resources and records it
in time series. Each time series created includes a description of the
monitored resource from which the data was collected.
Monitored-resource objects
A monitored resource is represented by an instance of a
MonitoredResource
object. The monitored resource describes
the source of the values in a time series.
The
MonitoredResource
object is embedded into the time series and
identifies a specific instance of a monitored-resource type known to
Monitoring: it points to a physical or logical entity.
For example, here is a monitored-resource object describing a particular
Compute Engine instance, extracted from the example time series:
"resource"
: {
"type": "gce_instance",
"labels": {
"instance_id": "5106847938295940291",
"zone": "us-central1-a",
"project_id": "a-gcp-project"
}
}
This instance of a monitored resource is of the type
gce_instance
, it has
a specific
instance_id
value (5106847938295940291) in the project
a-gcp-project
.
The instance is located in the US. Another time series might originate from a different
instance of this monitored resource. Each combination of label values identifies a
unique resource from which data is collected.
Different sets of labels make sense for different resources. The set of labels
for a resource is determined by the monitored-resource descriptor for that type.
Monitored-resource descriptors
The information about a monitored-resource type is held in a data structure
called a
monitored-resource descriptor
. For a definition of this
data structure, see
MonitoredResourceDescriptor
.
A monitored-resource descriptor is like a schema or specification for a record
structure. It doesn't contain the data about a particular instance of the
monitored resource; it tells you how to describe a monitored resource of a
specific type. For example, here is the monitored-resource descriptor for the
gce_instance
resource:
{
"type"
: "gce_instance",
"displayName": "VM Instance",
"description": "A virtual machine instance hosted in Compute Engine.",
"name"
: "projects/[PROJECT_ID]/monitoredResourceDescriptors/gce_instance"
"labels"
: [
{
"key": "project_id",
"description": "The identifier of the GCP project associated with this resource, such as \"my-project\"."
},
{
"key": "instance_id",
"description": "The numeric VM instance identifier assigned by Compute Engine."
},
{
"key": "zone",
"description": "The Compute Engine zone in which the VM is running."
}
],
}
Compare this monitored-resource descriptor to the specific instance of a
gce_instance
resource shown in
Monitored-resource
objects
.
This descriptor doesn't describe a particular monitored resource; instead,
the descriptor specifies the labels, and each combination of label values
identifies a particular monitored resource.
You can retrieve the monitored-resource descriptors in your Google Cloud project
by using the Cloud Monitoring API. See, for example, the reference pages for the
monitoredResourceDescriptors.list
and
monitoredResourceDescriptors.get
methods.
Live example:
If you have a Google Cloud project, you can retrieve this
descriptor by using the APIs Explorer widget as follows:
Open the
monitoredResourceDescriptors.list
reference page.
In the pane labeled
Try this method
, enter the following:
Click
Execute
.
The request returns the preceding descriptor.
For examples using the Monitoring API methods, see
List metric and resource types
.
Metrics
Each time series created by Cloud Monitoring records a set of data points
as well as information about the organization and meaning of those data points.
Metric objects
Each time series includes a reference to the description of the data
being recorded in a metric object.
The metric object contained in a time series specifies the type of measurements
and metric-specific information about those measurements.
For a definition of the metric
object data structure, see
Metric
.
For example, here is the metric object
extracted from the time series in
A sample time series
:
{
"metric"
: {
"labels": {
"log": "kubelet",
"severity": "DEFAULT"
},
"type": "logging.googleapis.com/log_entry_count"
}
This object tells you that the time series contains
logging.googleapis.com/log_entry_count
measurements. The label
values tell you that this specific time series counts only log entries
of severity
DEFAULT
in the log file called
kubelet
.
There is one time series for each combination of label values, so
INFO
entries to the same log file appear in a different time series.
The set of labels collected in a metric object is
specified in the descriptor for that metric type.
Metric descriptors
The information about a metric type is held in a data structure called
a
metric descriptor
. For a definition of this
data structure, see
MetricDescriptor
.
A metric descriptor is a schema or specification for a record structure. It
doesn't contain the data about a particular metric; rather, it tells you how to interpret
the data associated with a specific metric type.
Here is an example metric descriptor:
{
"type": "logging.googleapis.com/log_entry_count",
"name": "projects/a-gcp-project/metricDescriptors/logging.googleapis.com/log_entry_count",
"labels": [
{
"key": "log",
"description": "Name of the log."
},
{
"key": "severity",
"description": "Severity of the log entry."
}
],
"metricKind": "DELTA",
"valueType": "INT64",
"unit": "1",
"description": "Number of log entries that contributed to user-defined metrics.",
"displayName": "Log entries",
"metadata": {
"launchStage": "GA",
"samplePeriod": "60s"
},
"launchStage": "GA"
}
Compare this metric descriptor to the metric object shown in
Metric objects
. The descriptor tells you what the
label values mean, as well as how to interpret the values of the data points.
You can retrieve metric descriptors by using the Cloud Monitoring API. See, for
example, the reference pages for the
metricDescriptors.list
and
metricDescriptors.get
methods.
Live example:
If you have a Google Cloud project, you can retrieve this
descriptor by using the APIs Explorer widget as follows:
Open the
metricDescriptors.list
reference page.
In the pane labeled
Try this method
, enter the following:
- name
:
projects/
PROJECT_ID
/metricDescriptors/logging.googleapis.com/log_entry_count
Replace
PROJECT_ID
with the ID of your Google Cloud project.
Click
Execute
.
The request returns the preceding descriptor.
For examples using the Monitoring API methods, see
List metric and resource types
.
Most of the fields in the metric descriptor are self-explanatory. The
two likely to need further explanation are the metric kind and value type,
which are described further in
Metric kinds and types
.
Cloud Monitoring has approximately 6,500 built-in
metric types; see the
Metrics list
for details. You can also
create your own metric
descriptors to capture custom metrics. For more information, see
User-defined metrics overview
.