Firebase Test Lab for Android reduced American Express app test costs by 50%
Introduction
The AMEX mobile app, with a million Android downloads, is used by
Card Members to track spending and rewards, find offers, pay
bills, and enjoy app-only features. To facilitate frequent
updates, American Express built a device lab to test code changes
in their app across all platforms before release.
Challenge
With dozens of devices running various Android releases, the lab
was tough to scale. Adding new tests decreased productivity
because of the device lab's limited throughput, and when new team
members joined, the increased number of code changes added even
more load.
Maintaining the device lab was time-consuming and costly,
diverting the attention of engineers who otherwise would have
been focused on enhancements to drive additional value for Card
Members. The devices commonly had issues that required manual
intervention, often forcing reruns of the entire test suite.
Problematic devices were regularly removed from the device lab,
and replacing them often created down-time while the team awaited
the arrival of new devices.