Indexes are an important factor in the performance of a database. Much like the
index of a book which maps topics in a book to page numbers, a database index
maps the items in a database to their locations in the database. When you send a
database a query, the database can use an index to quickly look up the locations
of the items you requested.
This page describes the two types of indexes that Cloud Firestore uses,
single-field indexes
and
composite
indexes
.
Index definition and structure
An index is defined on a list of fields of a given document, with a corresponding
index mode
for each field.
An index contains an entry for every field named in the index's definition. The index includes all documents that are the potential results for queries based on the index. A document is included in the index only if it has an indexed value set for every field used in the index; if the index definition refers to a field for which the document has no value set, that document will not appear in the index and hence will never be returned as a result for any query based on the index.
The composite index is sorted by field values, in the order specified in the index definition.
An index behind every query
If no index exists for a query, most databases crawl through their contents item
by item, a slow process that slows down even more as the database grows.
Cloud Firestore guarantees high query performance by using indexes for
all
queries. As a result, query performance depends on the size
of the result set and not on the number of items in the database.
Less index management, more app development
Cloud Firestore includes features that reduce the amount of time you need
to spend managing indexes. The indexes required for the most basic queries are
automatically created for you. As you use and test your app,
Cloud Firestore helps you identify and
create additional
indexes
your app requires.
Index types
Cloud Firestore uses two types of indexes:
single-field
and
composite
. Besides the number of fields indexed, single-field and
composite indexes differ in how you manage them.
Single-field indexes
A single-field index stores a sorted mapping of all the documents in a
collection that contain a specific field. Each entry in a single-field index
records a document's value for a specific field and the location of the document
in the database. Cloud Firestore uses these indexes to perform many
basic queries. You manage single-field indexes by configuring your database's
automatic indexing settings and index exemptions.
Automatic indexing
By default, Cloud Firestore automatically maintains single-field indexes
for each field in a document and each subfield in a map. Cloud Firestore
uses the following default settings for single-field indexes:
For each non-array and non-map field, Cloud Firestore defines two
collection-scope
single-field indexes, one in ascending mode
and one in descending mode.
For each map field, Cloud Firestore creates the following:
- One collection-scope ascending index for each non-array, non-map subfield.
- One collection-scope descending index for each non-array, non-map subfield.
- One collection-scope array-contains index for each array subfield.
- Cloud Firestore recursively indexes each map subfield.
For each array field in a document, Cloud Firestore creates and
maintains a collection-scope array-contains index.
Single-field indexes with collection group scope are not maintained by
default.
Single-field index exemptions
You can exempt a field from your
automatic indexing
settings by creating a single-field index exemption.
An indexing exemption overrides the database-wide automatic index settings. An
exemption can enable a single-field index that your automatic indexing settings
would otherwise disable or disable a single-field index that automatic indexing
would otherwise enable. For cases where exemptions can be useful, see the
indexing best practices
.
Use the
*
field path value to add collection-level index exemptions on all
fields in a collection group. For example, for collection group
comments
, set
the field path to
*
to match all fields in the
comments
collection group and
disable indexing of all the fields under the collection group. You can then add
exemptions to index only the fields required for your queries. Reducing the
number of indexed fields reduces storage costs and can improve write
performance.
If you create a single-field index exemption for a map field, the map's
subfields inherit those settings. You can, however, define single-field index
exemptions for specific subfields. If you delete an exemption for a subfield,
the subfield will inherit its parent's exemption settings, if they exist, or the
database-wide settings if no parent exemptions exist.
To create and manage single-field index exemptions, see
Managing Indexes in Cloud Firestore
.
Composite indexes
A composite index stores a sorted mapping of all the documents in a collection,
based on an ordered list of fields to index.
Cloud Firestore uses composite indexes to support
queries not already supported by single-field indexes.
Cloud Firestore does not automatically create composite indexes like it
does for single-field indexes because of the large number of possible field
combinations. Instead, Cloud Firestore helps you
identify and create
required composite indexes
as you build your app.
If you attempt the query above without first creating the required index,
Cloud Firestore returns an error message containing a link you can follow
to create the missing index. This happens any time you attempt a query not
supported by an index. You can also define and manage composite indexes manually
by using the console or by using the
Firebase
CLI
. For more on creating and managing
composite indexes, see
Managing Indexes
.
Index modes and query scopes
You configure single-field and composite indexes differently, but both require
that you configure index modes and query scopes for your indexes.
Index modes
When you define an index, you select an index mode for each indexed field. Each
field's index mode supports specific query clauses on that field. You
can select from the following index modes:
Index mode
|
Description
|
Ascending
arrow_upward
|
Supports
<
,
<=
,
==
,
>=
,
>
,
!=
,
in
, and
not-in
, query clauses on the field and supports sorting results in ascending order based on this field value.
|
Descending
arrow_downward
|
Supports
<
,
<=
,
==
,
>=
,
>
,
!=
,
in
, and
not-in
query clauses on the field and supports sorting results in descending order based on this field value.
|
Array‑contains
|
Supports
array-contains
and
array-contains-any
query clauses on the field.
|
Vector
|
Supports
FindNearest
query clauses on the field.
|
Query scopes
Each index is scoped to either a collection or a collection group. This is known
as the index's query scope:
- Collection scope
- Cloud Firestore creates indexes with collection scope by default.
These indexes support queries that return results from a single collection.
- Collection group scope
- A collection group includes all collections with the same collection ID. To
run a
collection group query
that returns filtered
or ordered results from a collection group, you must create a corresponding
index with collection group scope.
Default ordering and the
__name__
field
In addition to sorting documents by the index modes
specified for each field (ascending or descending) , indexes apply a final
sorting by the
__name__
field of each document. The value of the
__name__
field is set to the full document path. This means that documents
in the result set with the same field values are sorted by document path.
By default, the
__name__
field is sorted in the same direction of the last
sorted field in the index definition. For example:
Collection
|
Fields indexed
|
Query scope
|
cities
|
arrow_upward
name,
arrow_upward
__name__
|
Collection
|
cities
|
arrow_downward
state,
arrow_downward
__name__
|
Collection
|
cities
|
arrow_upward
country,
arrow_upward
population,
arrow_upward
__name__
|
Collection
|
To sort results by the non-default
__name__
direction, you need to
create that index.
Perfect Index
The perfect index for a query, which allows the query to be executed most efficiently, is defined on the following properties, in order:
- Fields used in equality filters
- Fields used in sort orders
- Fields used in range & inequality filters (that are not already included in sort orders)
- Fields used in aggregations (that are not already included in sort orders and range & inequality filters)
Firestore computes the results for queries as follows:
- Identifies the index corresponding to the query's collection, filter properties, filter operators, and sort orders
- Identifies the index position to start scanning from using the query's equality filters and range & inequality filters on the first order by field
- Starts scanning the index, returning each document that satisfies all the filters, until it:
- encounters a document that does not meet the filter conditions and confirms that any subsequent document will never fully meet the filter conditions, or
- reaches the end of the index, or
- has collected the maximum number of results requested by the query
Indexing example
By automatically creating single-field indexes for you, Cloud Firestore
allows your application to quickly support the most basic database queries.
Single-field indexes allow you to perform simple queries based on field values
and the comparators
<
,
<=
,
==
,
>=
,
>
, and
in
. For array fields, they allow
you to perform
array-contains
and
array-contains-any
queries.
To illustrate, examine the following examples from the point of view of
index creation. The following snippet creates a
few
city
documents in a
cities
collection and sets
name
,
state
,
country
,
capital
,
population
, and
tags
fields for each document:
Web
var citiesRef = db.collection("cities");
citiesRef.doc("SF").set({
name: "San Francisco", state: "CA", country: "USA",
capital: false, population: 860000,
regions: ["west_coast", "norcal"] });
citiesRef.doc("LA").set({
name: "Los Angeles", state: "CA", country: "USA",
capital: false, population: 3900000,
regions: ["west_coast", "socal"] });
citiesRef.doc("DC").set({
name: "Washington, D.C.", state: null, country: "USA",
capital: true, population: 680000,
regions: ["east_coast"] });
citiesRef.doc("TOK").set({
name: "Tokyo", state: null, country: "Japan",
capital: true, population: 9000000,
regions: ["kanto", "honshu"] });
citiesRef.doc("BJ").set({
name: "Beijing", state: null, country: "China",
capital: true, population: 21500000,
regions: ["jingjinji", "hebei"] });
Assuming the default automatic indexing settings, Cloud Firestore updates
one ascending single-field index per non-array field, one descending single-
field index per non-array field, and one array-contains single-field index for
the array field. Each row in the following table represents an entry in a
single-field index:
Collection
|
Field indexed
|
Query scope
|
cities
|
arrow_upward
name
|
Collection
|
cities
|
arrow_upward
state
|
Collection
|
cities
|
arrow_upward
country
|
Collection
|
cities
|
arrow_upward
capital
|
Collection
|
cities
|
arrow_upward
population
|
Collection
|
cities
|
arrow_downward
name
|
Collection
|
cities
|
arrow_downward
state
|
Collection
|
cities
|
arrow_downward
country
|
Collection
|
cities
|
arrow_downward
capital
|
Collection
|
cities
|
arrow_downward
population
|
Collection
|
cities
|
array-contains
regions
|
Collection
|
Queries supported by single-field indexes
Using these automatically created single-field indexes, you can run simple
queries like the following:
Web
const stateQuery = citiesRef.where("state", "==", "CA");
const populationQuery = citiesRef.where("population", "<", 100000);
const nameQuery = citiesRef.where("name", ">=", "San Francisco");
You can also create
in
and compound equality (
==
) queries:
Web
citiesRef.where('country', 'in', ["USA", "Japan", "China"])
// Compound equality queries
citiesRef.where("state", "==", "CO").where("name", "==", "Denver")
citiesRef.where("country", "==", "USA")
.where("capital", "==", false)
.where("state", "==", "CA")
.where("population", "==", 860000)
If you need to run a compound query that uses a range comparison (
<
,
<=
,
>
, or
>=
) or if you need to sort by a different field, you must create a
composite index
for that query.
The
array-contains
index allows you to query the
regions
array field:
Web
citiesRef.where("regions", "array-contains", "west_coast")
// array-contains-any and array-contains use the same indexes
citiesRef.where("regions", "array-contains-any", ["west_coast", "east_coast"])
Queries supported by composite indexes
Cloud Firestore uses composite indexes to support
compound queries not already supported by single-field indexes. For example, you
would need a composite index for the following queries:
Web
citiesRef.where("country", "==", "USA").orderBy("population", "asc")
citiesRef.where("country", "==", "USA").where("population", "
<", 3800000)="" citiesref.where("country",="" "="="," "usa").where("population",="" "="">", 690000)
// in and == clauses use the same index
citiesRef.where("country", "in", ["USA", "Japan", "China"])
.where("population", ">", 690000)
",>
These queries require the composite index below. Since the query uses
an equality (
==
or
in
) for the
country
field, you can use
an ascending or descending index mode for this field. By default,
inequality clauses apply an ascending sort order based on the field in the
inequality clause.
Collection
|
Fields indexed
|
Query scope
|
cities
|
arrow_upward
(or
arrow_downward
) country,
arrow_upward
population
|
Collection
|
To run the same queries but with a descending sort order, you
need an additional composite index in the descending direction for
population
:
Web
citiesRef.where("country", "==", "USA").orderBy("population", "desc")
citiesRef.where("country", "==", "USA")
.where("population", "
<", 3800000)="" .orderby("population",="" "desc")="" citiesref.where("country",="" "="="," "usa")="" .where("population",="" "="">", 690000)
.orderBy("population", "desc")
citiesRef.where("country", "in", ["USA", "Japan", "China"])
.where("population", ">", 690000)
.orderBy("population", "desc")
",>
Collection
|
Fields indexed
|
Query scope
|
cities
|
arrow_upward
country,
arrow_upward
population
|
Collection
|
cities
|
arrow_upward
country
,
arrow_downward
population
|
Collection
|
You also need to create a composite index to combine an
array-contains
or
array-contains-any
query with additional clauses.
Web
citiesRef.where("regions", "array-contains", "east_coast")
.where("capital", "==", true)
// array-contains-any and array-contains use the same index
citiesRef.where("regions", "array-contains-any", ["west_coast", "east_coast"])
.where("capital", "==", true)
Collection
|
Fields indexed
|
Query scope
|
cities
|
array-contains
tags,
arrow_upward
(or
arrow_downward
) capital
|
Collection
|
Queries supported by collection group indexes
To demonstrate an index with collection group scope, imagine you add a
landmarks
sub-collection to some of the
city
documents:
Web
var citiesRef = db.collection("cities");
citiesRef.doc("SF").collection("landmarks").doc().set({
name: "Golden Gate Bridge",
category : "bridge" });
citiesRef.doc("SF").collection("landmarks").doc().set({
name: "Golden Gate Park",
category : "park" });
citiesRef.doc("DC").collection("landmarks").doc().set({
name: "National Gallery of Art",
category : "museum" });
citiesRef.doc("DC").collection("landmarks").doc().set({
name: "National Mall",
category : "park" });
Using the following single-field index with collection scope, you can query
a single city's
landmarks
collection based on the
category
field:
Collection
|
Fields indexed
|
Query scope
|
landmarks
|
arrow_upward
(or
arrow_downward
) category
|
Collection
|
Web
citiesRef.doc("SF").collection("landmarks").where("category", "==", "park")
citiesRef.doc("SF").collection("landmarks").where("category", "in", ["park", "museum"])
Now, imagine that you're interested in querying the landmarks across all cities.
To run this query on the collection group consisting of all
landmarks
collections, you must enable a
landmarks
single-field index with
collection group scope:
Collection
|
Fields indexed
|
Query scope
|
landmarks
|
arrow_upward
(or
arrow_downward
) category
|
Collection group
|
With this index enabled, you can query the
landmarks
collection group:
Web
var landmarksGroupRef = db.collectionGroup("landmarks");
landmarksGroupRef.where("category", "==", "park")
landmarksGroupRef.where("category", "in", ["park", "museum"])
To run a collection group query that returns filtered
or ordered results, you must enable a corresponding single-field or composite
index with collection group scope. Collection group queries that don't filter
or order results, however, do not require any additional index definitions.
For example, you can run the following collection group query without enabling
an additional index:
Web
db.collectionGroup("landmarks").get()
Index entries
Your project's configured indexes and the structure of a document determine
the number of index entries for a document. Index entries count towards
the
index entry count limit
.
The following example demonstrates the index entries of a document.
Document
/cities/SF
city_name : "San Francisco"
temperatures : {summer: 67, winter: 55}
neighborhoods : ["Mission", "Downtown", "Marina"]
Single-Field indexes
- city_name ASC
- city_name DESC
- temperatures.summer ASC
- temperatures.summer DESC
- temperatures.winter ASC
- temperatures.winter DESC
- neighborhoods Array Contains (ASC and DESC)
Composite indexes
- city_name ASC, neighborhoods ARRAY
- city_name DESC, neighborhoods ARRAY
Index entries
This indexing configuration results in the following 18 index entries for the
document:
Index
|
Indexed data
|
Single-field index entries
|
|
city_name ASC
|
city_name: "San Francisco"
|
city_name DESC
|
city_name: "San Francisco"
|
temperatures.summer ASC
|
temperatures.summer: 67
|
temperatures.summer DESC
|
temperatures.summer: 67
|
temperatures.winter ASC
|
temperatures.winter: 55
|
temperatures.winter DESC
|
temperatures.winter: 55
|
neighborhoods Array Contains ASC
|
neighborhoods: "Mission"
|
neighborhoods Array Contains DESC
|
neighborhoods: "Mission"
|
neighborhoods Array Contains ASC
|
neighborhoods: "Downtown"
|
neighborhoods Array Contains DESC
|
neighborhoods: "Downtown"
|
neighborhoods Array Contains ASC
|
neighborhoods: "Marina"
|
neighborhoods Array Contains DESC
|
neighborhoods: "Marina"
|
Composite index entries
|
|
city_name ASC, neighborhoods ARRAY
|
city_name: "San Francisco", neighborhoods: "Mission"
|
city_name ASC, neighborhoods ARRAY
|
city_name: "San Francisco", neighborhoods: "Downtown"
|
city_name ASC, neighborhoods ARRAY
|
city_name: "San Francisco", neighborhoods: "Marina"
|
city_name DESC, neighborhoods ARRAY
|
city_name: "San Francisco", neighborhoods: "Mission"
|
city_name DESC, neighborhoods ARRAY
|
city_name: "San Francisco", neighborhoods: "Downtown"
|
city_name DESC, neighborhoods ARRAY
|
city_name: "San Francisco", neighborhoods: "Marina"
|
Indexes and pricing
Indexes contribute to the
storage costs
of your application.
For more on how storage size for indexes is calculated, see
Index entry size
.
Taking advantage of index merging
Although Cloud Firestore uses an index for every query, it does not
necessarily require one index per query. For queries with multiple equality
(
==
) clauses and, optionally, an
orderBy
clause, Cloud Firestore can
re-use existing indexes. Cloud Firestore can merge the indexes for simple
equality filters to build the composite indexes needed for larger equality
queries.
You can reduce indexing costs by identifying situations where you can take
advantage of index merging. For example, imagine a
restaurants
collection for
a restaurant rating app:
Now, imagine this app uses queries like the ones below. Notice that the app uses
combinations of equality clauses for
category
,
city
, and
editors_pick
while always sorting by ascending
star_rating
:
Web
db.collection("restaurants").where("category", "==", "burgers")
.orderBy("star_rating")
db.collection("restaurants").where("city", "==", "San Francisco")
.orderBy("star_rating")
db.collection("restaurants").where("category", "==", "burgers")
.where("city", "==", "San Francisco")
.orderBy("star_rating")
db.collection("restaurants").where("category", "==", "burgers")
.where("city", "==" "San Francisco")
.where("editors_pick", "==", true )
.orderBy("star_rating")
You could create an index for each query:
Collection
|
Fields indexed
|
Query scope
|
restaurants
|
arrow_upward
category,
arrow_upward
star_rating
|
Collection
|
restaurants
|
arrow_upward
city,
arrow_upward
star_rating
|
Collection
|
restaurants
|
arrow_upward
category,
arrow_upward
city,
arrow_upward
star_rating
|
Collection
|
restaurants
|
arrow_upward
category,
arrow_upward
city,
arrow_upward
editors_pick,
arrow_upward
star_rating
|
Collection
|
As a better solution, you can reduce the number
of indexes by taking advantage of Cloud Firestore's ability to merge
indexes for equality clauses:
Collection
|
Fields indexed
|
Query scope
|
restaurants
|
arrow_upward
category,
arrow_upward
star_rating
|
Collection
|
restaurants
|
arrow_upward
city,
arrow_upward
star_rating
|
Collection
|
restaurants
|
arrow_upward
editors_pick,
arrow_upward
star_rating
|
Collection
|
Not only is this set of indexes smaller, it also supports an additional query:
Web
db.collection("restaurants").where("editors_pick", "==", true)
.orderBy("star_rating")
Indexing limits
The following limits apply to indexes. For all quotas and limits, see
Quotas and Limits
.
Limit
|
Details
|
Maximum number of composite indexes for a database
|
|
Maximum number of single-field configurations for a database
|
One field level configuration
can contain multiple configurations for the same field. For example,
a single-field indexing exemption and a TTL policy on the same field
count as one field configuration towards the limit.
|
Maximum number of index entries for each document
|
40,000
The number of index entries is the sum of the following for a document:
- The number of single-field index entries
- The number of composite index entries
To see how Cloud Firestore turns a document and a set of
indexes into index entries, see
this index entry count example
.
|
Maximum number of fields in a composite index
|
100
|
Maximum size of an index entry
|
7.5 KiB
To see how Cloud Firestore calculates index entry size, see
index entry size
.
|
Maximum sum of the sizes of a document's index entries
|
8 MiB
The total size is the sum of the following for a document:
The sum of the size of a document's single-field index entries
The sum of the size of a document's composite index entries
|
Maximum size of an indexed field value
|
1500 bytes
Field values over 1500 bytes are truncated. Queries involving
truncated field values may return inconsistent results.
|
Indexing best practices
For most apps, you can rely on automatic indexing and the error message links to
manage your indexes. However, you may want to add single-field exemptions in the
following cases:
Case
|
Description
|
Large string fields
|
If you have a string field that often holds long string values that
you don't use for querying, you can cut storage costs by exempting the field
from indexing.
|
High write rates to a collection containing documents with sequential values
|
If you index a field that increases or decreases sequentially between
documents in a collection, like a timestamp, then the maximum write rate to the
collection is 500 writes per second. If you don't query based on the field with sequential values, you can exempt the field
from indexing to bypass this limit.
In an IoT use case with a high write rate, for example, a collection containing documents with a timestamp field might approach the 500 writes per second limit.
|
TTL fields
|
If you use
TTL (time-to-live) policies
, note that the TTL
field must be a timestamp. Indexing on TTL fields is enabled by default and can
affect performance at higher traffic rates. As a best practice, add
single-field exemptions for your TTL fields.
|
Large array or map fields
|
Large array or map fields can approach the limit of 40,000 index entries per document. If you are not querying based on a large array or map field, you should exempt it from indexing.
|
If you are using queries with range & inequality operators on multiple fields, see the
indexing
considerations
that you should consider to optimize the
performance and cost of Cloud Firestore queries
For more information on how to resolve indexing issues (index fanout,
INVALID_ARGUMENT
errors) check out the
troubleshooting page
.