Manage
This page discusses the commonly-used metadata fields that are stored along with
objects in Cloud Storage.
Introduction
Objects stored in Cloud Storage have
metadata
associated with them.
Metadata identifies properties of the object, as well as specifies how the
object should be handled when it's accessed. Metadata exists as
key:value
pairs
. For example, the
storage class
of an object is represented
by the metadata entry
storageClass:STANDARD
.
storageClass
is the
key
for the metadata, and all objects have such a key associated with them.
STANDARD
specifies the
value
this specific object has, and the value varies
from object to object.
The mutability of metadata varies: some metadata you can edit at any time, some
metadata you can only set at the time the object is created, and some metadata
you can only view. For example, you can edit the value of the
Cache-Control
metadata at any time, but you can only assign the
storageClass
metadata when
the object is created or rewritten, and you cannot directly edit the value
for the
generation
metadata, though the
generation
value changes when
the object is replaced.
Editable metadata
There are two categories of metadata that
users can change
for objects:
Fixed-key metadata
: Metadata whose keys are set, but for which you
can specify a value.
Custom metadata
: Metadata that you add by specifying both a key and
a value associated with the key.
When editing metadata, you should generally avoid non-ascii characters,
because they are not permitted in HTTP headers, which the XML API uses.
Fixed-key metadata
You can edit the following metadata for objects, though you must
have sufficient
permission
to do so:
Access control metadata
Cloud Storage uses
Identity and Access Management (IAM)
and
Access Control Lists (ACLs)
to control access to objects. Use these links
to learn about these access control methods and associated metadata.
Cache-Control
The
Cache-Control
metadata can specify two different aspects
of how data is served from Cloud Storage: whether the data can be
cached and whether the data can be transformed.
Caching data
The
Cache-Control
metadata lets you control whether and for how long
browser and Internet caches are allowed to cache your objects, which can then
be served to satisfy future requests. The Cloud Storage network only
considers an object's
Cache-Control
setting when accessing objects that:
Cloud Storage respects
standard values
for
Cache-Control
, such as the following:
public
: the object can be cached anywhere.
private
: the object can be cached in a requester's local cache.
no-cache
: the object can be cached, but cannot be used to satisfy future
requests unless first validated by Cloud Storage.
no-store
: the object can't be cached.
max-age=
TIME_IN_SECONDS
: the length of time an object
may be cached before it's considered stale. You can set
max-age
to any
length of time. Stale objects are not served from caches, except in
special
circumstances
.
If an applicable object doesn't have a
Cache-Control
metadata entry,
Cloud Storage uses the default value of:
public, max-age=3600
, if the object is not encrypted using a
customer-managed encryption key
or stored within a
Virtual Private Cloud service perimeter
.
no-cache, no-store, max-age=0
, if the object is encrypted using a
customer-managed encryption key
.
private, max-age=0
, if the object is stored within a
Virtual Private Cloud service perimeter
.
no-cache, no-store, max-age=0, must-revalidate
, if the object is stored
within a
Virtual Private Cloud service perimeter
and
encrypted using a
customer-managed encryption key
.
If you allow caching, downloads may continue to receive earlier versions of
an object, even after uploading a newer version. This is because the earlier
version remains "fresh" in the cache for a period of time determined by
max-age
. Additionally, because objects can be cached at various places on the
Internet, there is no way to force a cached object to expire globally. If you
want to prevent serving cached versions of publicly readable objects, set
Cache-Control: no-store
on the object.
For more information about caching with Cloud Storage and
Cloud CDN, see
Caching
.
If you need greater control over cache behavior, you can configure
Cloud CDN in front of your bucket
.
Cache-Control
metadata also lets you serve objects as they are stored,
without applying any
transformations
to the data, such as removing gzip
content-encoding for incompatible clients. To serve an object as-is, set
Cache-Control:no-transform
.
Content-Disposition
The
Content-Disposition
metadata specifies presentation
information about the data being transmitted. Setting
Content-Disposition
allows you to control presentation style of the content, for example
determining whether an attachment should be automatically displayed or whether
some form of action from the user should be required to open it. See
https://datatracker.ietf.org/doc/html/rfc6266
for the
Content-Disposition
specification.
Content-Encoding
The
Content-Encoding
metadata can be used to indicate that an object is
compressed, while still maintaining the object's underlying
Content-Type
.
For example, a text file that is gzip compressed can have the fact that
it's a text file indicated in
Content-Type
and the fact that it's
gzip compressed indicated in
Content-Encoding
. You should ensure that
files are, in fact, compressed using the specified
Content-Encoding
before
uploading them, or else unexpected behavior can occur when attempting to
download the objects. For more information, see the
Transcoding page
.
For compressible content, such as text, using
Content-Encoding: gzip
saves
network and storage costs and improves content serving performance. However,
for content that is already inherently compressed, such as archives and many
media formats, applying another level of compression and marking it in the
Content-Encoding
metadata is typically detrimental to both object size and
performance and should be avoided.
Content-Language
The
Content-Language
metadata indicates the language(s) that
the object is intended for. Refer to
ISO 639-1
language codes
for typical values of this metadata.
Cloud Storage supports
Content-Language
values up to 100 characters
in length.
Content-Type
The most commonly set metadata is
Content-Type
(also known as media type),
which lets browsers render the object properly. All objects have a value
specified in their
Content-Type
metadata, but this value does not have to
match the underlying type of the object. For example, if the
Content-Type
is
not specified by the uploader and cannot be determined, it is set to
application/octet-stream
or
application/x-www-form-urlencoded
, depending on
how you uploaded the object. For a list of valid content types, see the
IANA Media Types
page.
Custom-Time
The
Custom-Time
metadata is a user-specified date and time represented in
the
RFC 3339
format
YYYY-MM-DD'T'HH:MM:SS.SS'Z'
or
YYYY-MM-DD'T'HH:MM:SS'Z'
when milliseconds are zero. This metadata is
typically set in order to use the
DaysSinceCustomTime
condition in
Object Lifecycle Management.
You cannot remove
Custom-Time
once it's been set on an object. Additionally,
the value for
Custom-Time
cannot decrease. That is, you cannot set
Custom-Time
to be an earlier date/time than the existing
Custom-Time
. You
can, however, effectively remove or reset the
Custom-Time
by
rewriting the object.
Object holds
Use metadata flags to place object holds, which prevent objects from being
deleted or replaced. For more information, see the
Object holds page
.
Retention configuration
When present, an object's retention configuration defines a date and time prior
to which the object cannot be deleted or replaced. For more information, see
Object Retention Lock
.
Custom metadata is metadata for which you define both the key and the value. To
create custom metadata, you specify both a key and a value. After you have
created a custom metadata
key:value
pair, you can delete the key or change the
value.
Custom metadata is subject to a
size limit
and incurs
storage costs
.
The
Viewing and Editing Metadata page
includes information about setting
custom metadata.
The
XML API
sets and retrieves object metadata using request headers, and
the
JSON API
allows setting custom metadata in the final request of a
resumable upload
by using request headers. In order to clearly distinguish
custom metadata headers from standard request headers, both APIs prefix such
custom metadata headers with
x-goog-meta-
.
Non-editable metadata
Some metadata cannot be edited directly. This metadata is set at the time of
object creation or rewrite. As part of the object creation or rewrite,
you can set some such metadata, such as the
storage class
of the object
or
customer-managed encryption keys
. Other metadata is automatically
added and can only be viewed, such the generation number of the object or
the time of creation.
Generation and metageneration numbers
As part of its metadata, every Cloud Storage object has a numeric
generation
property and a numeric
metageneration
property that uniquely
identifies the object:
Property
|
Description
|
generation
|
Identifies the version of an object, and exists for every object, regardless of whether a bucket uses
Object Versioning
.
- An object version's
generation
value never changes. A new object with the same name can replace an existing object, but the new object always has a different
generation
assigned to it.
- There is no guarantee that generation numbers increase for successive versions, only that each new version has a unique generation number.
- There is no relationship between the generation numbers of unrelated objects, even if the objects are in the same bucket.
|
metageneration
|
Identifies the metadata version, and increases every time the metadata
of a given
generation
is updated.
metageneration
begins at
1
for each new
generation
of an object.
- The
metageneration
property has no meaning without the
generation
property and should be used only in conjunction with it; it's meaningless to compare metadata generations of two object versions.
|
The
generation
and
metageneration
properties are used in the following
circumstances:
Modification time
As part of its metadata, every Cloud Storage object has an
updated
property that indicates the last time the object's metadata was modified. The
updated
time is set initially to the object's creation time and then changes
whenever any metadata of the object changes. This includes changes made by a
requester, such as modifying custom metadata, as well as changes made by
Cloud Storage on behalf of a requester, such as changing the storage
class based on an
Object Lifecycle Configuration
.
What's next