This page describes how to use point-in-time recovery (PITR) to retain and recover data in Cloud Firestore.
To understand PITR concepts, see
Point-in-time recovery
.
Permissions
To get the permissions that you need to manage PITR settings, ask your administrator to grant you the following IAM roles on the project where you want to enable PITR:
- Cloud Datastore Owner (
roles/datastore.owner
)
For custom roles, ensure that the following permissions are granted:
- To enable PITR when creating a database:
datastore.databases.create
- To update PITR settings on existing database:
datastore.databases.update
,
datastore.databases.list
- To perform reads from PITR data:
datastore.databases.get
,
datastore.entities.get
,
datastore.entities.list
- To export PITR data:
datastore.databases.export
- To import PITR data:
datastore.databases.import
Before you begin
Note the following points before you start using PITR:
- You can't start reading from seven days in the past immediately after you
enable PITR.
- If you want to enable PITR when you create a database, you must use the
gcloud firestore databases create
command. Enabling PITR while
creating a database using the GCP Console is not supported.
- Cloud Firestore starts retaining versions from the point forward after
enabling PITR.
- You cannot read PITR data in the PITR window after you disable PITR.
- If you re-enable PITR immediately after disabling it, the past PITR data
is no longer available. Any PITR data created before disabling PITR will
be deleted after the PITR expiration date.
- If you accidentally deleted data in the last hour and PITR is disabled, you
can restore your data by enabling PITR within one hour of deletion.
- Any read performed on expired PITR data fails.
Enable PITR
Before using PITR,
enable billing for your Google Cloud
project
.
Only Google Cloud projects with billing enabled can use the PITR functionality.
To enable PITR for your database:
Console
In the Google Cloud Platform Console, go to the
Databases
page.
Go to Databases
Select the required database from the list of databases.
In the navigation menu, click
Disaster Recovery
.
Click
Edit
to edit the settings.
Select the
Enable point-in-time recovery
check box, and then click
Save
.
Enabling PITR would incur storage costs. See
Pricing
for more information.
To disable PITR, clear the
Enable point-in-time recovery
check box from the Disaster Recovery page in the GCP Console.
gcloud
Enable PITR during database creation with the
gcloud firestore databases create
command as follows:
gcloud firestore databases create\
--location=
LOCATION
\
[--database=
DATABASE_ID
; default="(default)"]\
[--type=
TYPE
; default="firestore-native"]\
--enable-pitr
Replace the values as follows:
Location
- location where you want to create your database.
DATABASE_ID
- set to the database ID or (default).
TYPE
- set to firestore-native.
You can disable PITR using the
gcloud firestore databases update
command as follows:
gcloud firestore databases update\
[--database=
DATABASE_ID
; default="(default)"]\
--no-enable-pitr
Replace the values as follows:
DATABASE_ID
- set to the database ID or (default).
Get the retention period and earliest version time
Console
In the Google Cloud Platform Console, go to the
Databases
page.
Go to Databases
Select the required database from the list of databases.
In the navigation menu, click
Disaster Recovery
.
In the
Settings
section, note the
Retention period
and
Earliest version time
.
- Retention period
: the period in which Cloud Firestore retains
all versions of data for the database. The value is one hour when PITR is
disabled and seven days when PITR is enabled.
- Earliest version time
: the earliest timestamp at which older versions of
the data can be read in the PITR window. This value is continuously updated
by Cloud Firestore and becomes stale the moment it is queried. If you
are using this value to recover data, make sure to account for the time from
the moment w?hen the value is queried to the moment when you initiate the
recovery.
- Point-in-time recovery
: shows
Enabled
, if PITR
is enabled. If PITR is disabled, you will see
Disabled
.
gcloud
Run the
gcloud firestore databases describe
command as follows:
gcloud firestore databases describe --database=
DATABASE_ID
Replace
DATABASE_ID
with the database ID or
default
.
Here's the output:
appEngineIntegrationMode: ENABLED
concurrencyMode: PESSIMISTIC
createTime: '2021-03-24T17:02:35.234Z'
deleteProtectionState: DELETE_PROTECTION_DISABLED
earliestVersionTime: '2023-06-12T16:17:25.222474Z'
etag: IIDayqOevv8CMNTvyNK4uv8C
keyPrefix: s
locationId: nam5
name: projects/PROJECT_ID/databases/(default)
pointInTimeRecoveryEnablement: POINT_IN_TIME_RECOVERY_DISABLED
type: FIRESTORE_NATIVE
uid: 5230c382-dcd2-468f-8cb3-2a1acfde2b32
updateTime: '2021-11-17T17:48:22.171180Z'
versionRetentionPeriod: 3600s
where,
earliestVersionTime
- timestamp of the earliest PITR data stored.
pointInTimeRecoveryEnablement
: shows
POINT_IN_TIME_RECOVERY_ENABLED
, if PITR
is enabled. If PITR is disabled, you will either see
POINT_IN_TIME_RECOVERY_DISABLED
or the
pointInTimeRecoveryEnablement
field might not be displayed.
versionRetentionPeriod
- time period for which PITR data is retained in milliseconds. The value can be one hour when PITR is disabled or seven days if PITR is enabled.
Read PITR data
You can read PITR data using the client libraries, REST API methods, or FirestoreIO Apache Beam connector.
Client libraries
Java
You must use the
ReadOnly
transaction to read PITR data. You cannot directly specify
readTime
in reads.
See
Transactions and batched writes
for more information.
Firestore firestore = …
TransactionOptions options =
TransactionOptions.createReadOnlyOptionsBuilder()
.setReadTime(
com.google.protobuf.Timestamp.newBuilder()
.setSeconds(1684098540L)
.setNanos(0))
.build();
ApiFuture<Void> futureTransaction = firestore.runTransaction(
transaction -> {
// Does a snapshot read document lookup
final DocumentSnapshot documentResult =
transaction.get(documentReference).get();
// Executes a snapshot read query
final QuerySnapshot queryResult =
transaction.get(query).get();
},
options);
// Blocks on transaction to complete
futureTransaction.get();
Node
You must use a
ReadOnly
transaction to read PITR data. You cannot directly specify
readTime
in reads.
See
Transactions and batched writes
for more information.
const documentSnapshot = await firestore.runTransaction(
updateFunction => updateFunction.get(documentRef),
{readOnly: true, readTime: new Firestore.Timestamp(1684098540, 0)}
);
const querySnapshot = await firestore.runTransaction(
updateFunction => updateFunction.get(query),
{readOnly: true, readTime: new Firestore.Timestamp(1684098540, 0)}
)
REST API
PITR reads are supported in all Cloud Firestore read methods, which are
get
,
list
,
batchGet
,
listCollectionIds
,
listDocuments
,
runQuery
,
runAggregationQuery
, and
partitionQuery
.
To perform a read using the REST methods, try one of the following options:
In the your read method request, pass the
readTime
value as as a supported PITR timestamp in the
readOptions
method. A PITR timestamp can be either microsecond precision timestamp within the past hour or a whole minute timestamp beyond the past hour, but not earlier than the
earliestVersionTime
.
Use the
readTime
parameter together with the
BeginTransaction
method as part of a
ReadOnly
transaction
for multiple PITR reads.
Apache Beam
Use the Cloud FirestoreIO Apache Beam connector to read or write documents in a Cloud Firestore database at a large scale with Dataflow.
PITR reads are supported in the following read method of the
Cloud FirestoreIO connector. These read methods support the
withReadTime(@Nullable Instant readTime)
method that you use can use for PITR
reads:
Java
The following code can be used with the
example Dataflow pipeline code
for bulk read or write operations. The example uses the
withReadTime(@Nullable Instant readTime)
method for PITR reads.
Instant readTime = Instant.ofEpochSecond(1684098540L);
PCollection<Document> documents =
pipeline
.apply(Create.of(collectionId))
.apply(
new FilterDocumentsQuery(
firestoreOptions.getProjectId(), firestoreOptions.getDatabaseId()))
.apply(FirestoreIO.v1().read().runQuery().withReadTime(readTime).withRpcQosOptions(rpcQosOptions).build())
...
For a complete list of
readTime
examples in the Dataflow pipeline, see the
Github repository
.
Export and import from PITR data
You can export your database to Cloud Storage from PITR data
using the
gcloud firestore export
command. You can export PITR data where the timestamp is a whole minute timestamp within
the past seven days, but not earlier than the
earliestVersionTime
. If data no longer
exists at the specified timestamp, the export operation fails.
The PITR export operation supports all filters, including export of all documents and export of specific collections.
Export the database, specifying the
snapshot-time
parameter to the desired recovery timestamp.
gcloud
Run the following command to export the database to your bucket.
gcloud firestore export gs://[BUCKET_NAME_PATH] \
--snapshot-time=[
PITR_TIMESTAMP
] \
--collection-ids=[
COLLECTION_IDS
] \
--namespace-ids=[
NAMESPACE_IDS
]
Where,
BUCKET_NAME_PATH
- a valid Cloud Storage bucket with an optional path prefix where export files are stored.
PITR_TIMESTAMP
- a PITR timestamp at the minute granularity, for example,
2023-05-26T10:20:00.00Z
or
2023-10-19T10:30:00.00-07:00
.
COLLECTION_IDS
- a list of collection IDs or collection group IDs, for example-
'specific collection group1'
,
'specific
collection group2'
.
NAMESPACE_IDS
- a list of namespace IDs, for example-
'customer'
,
'orders'
.
Note the following points before exporting PITR data:
- Specify the timestamp in
RFC 3339
format
.
For example,
2023-05-26T10:20:00.00Z
or
2023-10-19T10:30:00.00-07:00
.
- Make sure that the timestamp you specify is a whole minute timestamp
within the past seven days, but not earlier than the
earliestVersionTime
. If data no longer exists at the specified
timestamp, an error is generated. The timestamp must be a whole minute, even if the specified time is within the past hour.
- You are not charged for a failed PITR export.
Import to a database.
Use the steps in
Import all documents
to import your
exported database. If any document already exists in your database, it will
be overwritten.