Personalization uses machine learning—specifically a
contextual multi-armed bandit algorithm—to
determine the optimal experience for individual users to achieve an objective.
In our case, the objective is to optimize for the total number or the total
parameter value of specific Google Analytics events.
What's a contextual multi-armed bandit algorithm?
The "multi-armed bandit" is a metaphor used to describe the situation where we
want to continually choose a path that leads to the highest, most reliable
rewards from a list of multiple paths. To visualize this, you can use the
metaphor of a gambler in front of a row of slot machines?often colloquially
referred to as a "one-armed bandit" because a slot machine has one handle (or
arm) and takes your money. Since we want to solve for multiple
"arms," the one-armed bandit becomes the
multi-armed
bandit.
For example, say we have three options and we want to determine which provides
the most reliable reward: We could try each option, and then, after receiving a
result, we could just keep choosing the arm that yielded the most rewards. This
is what's referred to as a
greedy
algorithm: the option that yields the best
result when we first attempt it is the one we'll continue to choose. But we can
understand that this might not always work—for one thing, the high reward
could be a fluke. Or maybe there's some user-specific context that resulted in
higher rewards during that time period that wouldn't be as effective later.
So
context
is added to make the algorithm more effective. For
Remote Config personalization, this initial context is random sampling,
or
uncertainty
, that provides some entropy to the experiment. This implements
a "
contextual
multi-armed bandit." As the experiment continues to run,
ongoing exploration and observation adds real learned context about which arms
are most likely to elicit a reward to the model, making it more effective.
What does this mean for my app?
Now, let's discuss what a multi-armed bandit algorithm means in the context of
your app. Let's say you're optimizing for banner ad clicks. In this case, the
"arms" of the personalization would be the
alternative values
you specify to
represent the different banner ads you want to display to users. The banner ad
click is the reward, which we refer to as an
objective
.
When you first launch a personalization, the model does not know which
alternative value will be more likely to achieve your goal for each individual
user. As the personalization explores each alternative value to understand the
likelihood of achieving your objective, the underlying model grows more
informed, improving its ability to predict and select the optimum experience for
each user.
Personalization uses a
stickiness window
of 24 hours. This is the amount of
time the personalization algorithm explores a single alternative value. You
should provide your personalizations enough time to explore each alternative
value multiple times (generally about 14 days). Ideally, you can let them run
perpetually so that they can continually improve and adapt as your app and user
behaviors change.
Track additional metrics
Remote Config personalization also provides the ability to track up to two
additional metrics, to help you contextualize your results. Let's say you've
developed a social app and have set different alternative values to encourage
users to share content with friends to increase overall engagement.
In this case, you might choose to optimize for an Analytics event like
link_received
and set your two metrics to
user_engagement
and
link_opened
to understand whether user engagement and the number of links the
user opens rises (true engagement) or falls (possibly too many spammy links).
While these additional metrics won't be factored into the personalization
algorithm, you can track them right alongside your personalization results,
providing valuable insight into the personalization's ability to achieve your
overall goals.
Understand personalization results
After a personalization has been running for long enough to gather data, you can
view its results.
To view personalization results:
Open the
Remote Config page
and click
Personalizations
.
Select the personalization you want to view. You can search for the
specific personalization by name or objective, and can sort by Name,
Start time, or Total lift.
The results page summarizes the
Total lift
, or percentage difference in
performance, that the personalization provides over the
Baseline
group.
The results page also shows the current status of the personalization, the
attributes of the personalization, and an interactive graph that:
Shows a detailed daily and total view of how the personalization performed
against the baseline.
Shows how each value performs overall across the baseline group.
Displays goal outcomes and performance against the additional metrics you
chose, accessible using the tabs at the top of the summary.
A personalization can be left running indefinitely and you can continue to
revisit the results page to monitor its performance. The algorithm will continue
to learn and adjust, so that it can adapt when user behavior changes.
Understand personalization deletion
You can delete a personalization using the Firebase console or by removing a
personalization parameter from your template using the
Firebase Remote Config API
.
Deleted personalizations cannot be restored. To learn about data retention, see
Data deletion
.
You can also delete personalizations by
rolling
back
or
importing a
template
.
Rollbacks
If your current template has personalizations and you
roll
back
to a template that does not have
the same personalizations, the personalizations are deleted. To revert to a
previous template, use the Firebase console or
roll back
using the Firebase Remote Config API.
When you delete a personalization and roll back to a previous template, a
reference to that invalid personalization appears in the Firebase console.
You can remove the invalid personalization from the
Firebase console
by editing the
personalization in the Parameters tab of the Remote Config page.
Imports
Importing a template that no longer contains your current personalizations also
deletes those personalizations. To import a template,
use the Firebase console
or use the
Remote Config REST API
.
Next steps