To run CTS, first prepare your physical environment, your desktop machine, and
the Android device you're using for testing.
Physical environment
Bluetooth LE beacons
If the device under test (DUT) supports Bluetooth LE, place at least three
Bluetooth LE beacons within 5?meters of the DUT for Bluetooth LE scan testing.
Those beacons don't need to be configured or emit anything specific, and can be
any kind, including iBeacon, Eddystone, or even devices simulating BLE beacons.
Ultra-wideband
If the DUT supports ultra-wideband (UWB), another device
supporting UWB must be positioned close enough and oriented so as not to have
an antennae and radio dead zone. For the distance accuracy tests, there are
specific positioning and orientation needs. For setup details, see
UWB requirements
.
The UWB test must be run manually, specifying on the command line which two
devices are one meter apart.
For details on sharding which is required for this test, see
Local sharding
.
Cameras
When running camera CTS, use normal lighting conditions with a test pattern
chart (such as a checkerboard pattern). Place the test pattern chart according
to the DUT's minimum focus distance to ensure that it isn't too close to the
lens.
Point the camera sensors to a scene with sufficient lighting to allow the
sensors under test to reach and remain at the maximum configured target frames
per second (FPS) as specified in
CONTROL_AE_TARGET_FPS_RANGE
.
This applies to all camera sensors reported by
getCameraIdList
as the test iterates over the listed devices and measures performance
individually.
If the DUT supports external cameras, such as USB webcams, plug in an external
camera when running CTS. Otherwise, the CTS tests fail.
GPS/GNSS
If the DUT supports the global positioning system/global navigation satellite
system (GPS/GNSS) feature, provide a GPS/GNSS signal to the DUT at a suitable
signal level for reception and GPS location calculation. The GPS portion must be
compliant with ICD-GPS-200C. Otherwise, the GPS/GNSS signal can be of any kind,
including a satellite simulator or a GPS/GNSS repeater of outdoor signals, or
you can place the DUT close enough to a window such that it can directly receive
enough GPS/GNSS signal.
Wi-Fi and IPv6
CTS tests require a Wi-Fi network that supports IPv4 and IPv6, has an internet
connection with working DNS for IPv4 and IPv6, supports IP multicast, and can
treat the DUT as an isolated client.
An
isolated client
is a configuration where the DUT doesn't have
visibility to the broadcast/multinetwork messages on that subnetwork. This
occurs with a Wi-Fi access point (AP) configuration or by running the DUT on an
isolated subnetwork without other devices being connected.
If you don't have access to a native IPv6 network, an IPv6 carrier network, or a
VPN to pass some tests depending on IPv6, you can use a Wi-Fi access point and
an IPv6 tunnel.
To pass CTS, the DUT needs the
UP
,
BROADCAST
, and
MULTICAST
flags set on
the Wi-Fi interface. The Wi-Fi interface needs IPv4 and IPv6 addresses assigned.
Check the Wi-Fi interface properties with
adb shell ifconfig
.
For devices that support
Wi-Fi STA/STA Concurrency
,
multiple Wi-Fi networks (at least 2) are required. To pass CTS, the Wi-Fi
networks must run on different bands with different SSIDs or on the
same SSID with different BSSIDs.
Wi-Fi RTT
Android includes the
Wi-Fi RTT API
for a
Wi-Fi round trip time (RTT)
capability. This allows devices to measure their distance to access points with
an accuracy of 1 to 2?meters, significantly increasing indoor location accuracy.
Two recommended devices supporting Wi-Fi RTT are
Google Wifi
and
Compulab's fitlet2 access point
(set to 40?MHz bandwidth at 5?GHz).
The access points should be powered up, but don't require a network connection.
Access points don't need to be next to the testing device but are recommended to
be within 40?feet of the DUT. One access point is typically sufficient.
Desktop machine setup
Caution
: CTS supports 64-bit Linux machines. CTS is not supported on Windows OS
or MacOS.
FFMPEG
Install the ffmpeg version 5.1.3 (or later) package on the host machine.
Host machine upgrade
It is highly recommended to upgrade the CTS host machine RAM to 128GB and HDD to 256GB. It is required to accommodate the increased number of CTS test cases and an increase in Java heap space reservation in tradefed.
ADB and AAPT2
Before running the CTS, ensure that you have installed the recent versions of
both
Android Debug Bridge (adb)
and
Android Asset Packaging Tool (AAPT2)
and added the location of those tools to the system path of your machine.
To install ADB and AAPT2, download the latest
Android SDK Platform Tools
and
Android SDK Build Tools
from Android Studio's
SDK Manager
or from the
sdkmanager
command line tool.
Ensure that
adb
and
aapt2
are in your system path. The following command
assumes that you've downloaded the package archives to a sub-directory called
android-sdk
in your home directory:
export PATH=$PATH:$HOME/android-sdk/platform-tools:$HOME/android-sdk/build-tools/<tools version number>
Java Development Kit for Ubuntu
Install the proper version of
Java Development Kit (JDK)
.
- For Android 11, install OpenJDK11.
- For Android 9 and Android 10,
install OpenJDK9.
- For Android 7.0, 7.1, 8.0 and 8.1, install OpenJDK8.
For details, see the
JDK requirements
.
Setup for Python support
Install
virtualenv
for your platform by following the
Installation
instructions.
You can verify that the installation is successful by invoking
virtualenv -h
.
CTS files
Download and open the CTS packages from
Compatibility Test Suite Downloads
matching your devices' Android version and all the application binary interfaces
(ABIs) that your devices support.
Download and open the latest version of the
CTS media files
.
Device detection
Follow the step to
set up your system to detect your device
.
Memory limit
You might want to increase the maximum memory available during test run in the
cts-tradefed
script. Refer to
example CL
for more information.
Android device setup
User builds
A compatible device is defined as a device with a user/release-key signed build.
Your device should be running a system image based on the known to be compatible
user build (Android 4.0 or higher) from
Codenames, Tags, and Build Numbers
.
First API level build property
Certain CTS requirements depend on the build that a device was originally
shipped with. For example, devices that originally ship with earlier builds
might be excluded from system requirements that apply to devices that ship with
later builds.
To make this information available to CTS, device manufacturers could have
defined the build-time property
ro.product.first_api_level
. The value of this
property is the first API level that the device was commercially launched with.
The device manufacturers can reuse the common underlying implementation to
launch a new product as an upgrade of an existing product in the same device
group. The device manufacturers can optionally set the API level of the existing
product to
ro.product.first_api_level
, so that upgrade requirements are
applied for CTS and Treble/VTS.
The device manufacturers can define
PRODUCT_SHIPPING_API_LEVEL
in their
device.mk
file to set this property, as shown in the following example:
# PRODUCT_SHIPPING_API_LEVEL sets ro.product.first_api_level to indicate
# the first api level that the device has been commercially launched on.
PRODUCT_SHIPPING_API_LEVEL := 21
First API level for Android 9 or higher
For devices launched with Android 9 or higher, set the
ro.product.first_api_level
property to a valid value from
Codenames, Tags, and Build Numbers
.
First API level for Android 8.x or lower
For devices launched on Android 8.x or lower, unset (remove) the
ro.product.first_api_level
property for the first build of the product. For
all subsequent builds, set
ro.product.first_api_level
to the correct API level
value. This allows the property to correctly identify a new product and
preserves information about the first API level of the product. If the flag is
unset, Android assigns
Build.VERSION.SDK_INT
to
ro.product.first_api_level
.
CTS shim packages
Android 10 or higher includes a package format called
APEX
. To run CTS tests for APEX management
APIs (such as updating to a new version or reporting active APEXes) you must
preinstall a
CtsShimApex
package on a
/system
partition.
The APEX shim validation test verifies the implementation of
CtsShimApex
.
ro.apex.updatable requirements
If the
ro.apex.updatable
property is set to
true
,
CtsShimApex
is
required for all devices that support APEX package management.
If the
ro.apex.updatable
property is missing or isn't set,
CtsShimApex
isn't required to be preinstalled on a device.
The APEX shim validation test verifies the implementation of
CtsShimApex
.
CtsShim preinstalls and preloads
Starting with Android 11,
CtsShimApex
contains two
prebuilt apps (built from
build source
),
which don't contain any code except for the manifest. CTS uses these apps to
test privileges and permissions.
If the device doesn't support APEX package management (that is, the
ro.apex.updatable
property is missing or isn't set), or if the device is
running version 10 or lower, the two prebuilt apps must
be preinstalled in the system separately.
If APEX is supported, the preinstalls for the appropriate release must be placed as
/system/apex/com.android.apex.cts.shim.apex
.
If regular prebuilt apps are used,
CtsShim
and
CtsShimPriv
for the
appropriate release must be placed as
/system/app/CtsShimPrebuilt.apk
and
/system/priv-app/CtsShimPrivPrebuilt.apk
respectively.
The following table lists the preinstalls and preloads available for each
device version and architecture.
To pass the tests, preload the apps into the appropriate directories on the
system image without re-signing the apps.
Sample applet
Android 9 introduced Open Mobile APIs. For devices that report more than one
secure element, CTS adds test cases to validate the behavior of the Open Mobile
APIs. These test cases require the one-time installation of a sample applet into
the embedded Secure Element (eSE) of the DUT or into the SIM card used by the
DUT. The
eSE sample applet
and the
SIM sample applet
can be found in AOSP.
See
CTS Test for Secure Element
for
more detailed information on Open Mobile API test cases and Access Control test
cases.
Storage requirements
The CTS media stress tests require video clips to be on external storage
(
/sdcard
). Most of the clips are from
Big Buck Bunny
, which is copyrighted
by the Blender Foundation under the
Creative Commons Attribution 3.0 license
.
The required space depends on the maximum video playback resolution supported by
the device. See section 5 in the
Android Compatibility Definition document
for the
platform version of the required resolutions.
Here are the storage requirements by maximum video playback resolution:
- 480x360: 98?MB
- 720x480: 193?MB
- 1280x720: 606?MB
- 1920x1080: 1863?MB
Screen and storage
- Any device that doesn't have an embedded screen needs to be connected to a
screen.
If the device has a memory card slot, plug in an empty SD card.
Use an SD
card that supports ultra high speed (UHS) bus with SDHC or SDXC capacity or
one with at least speed class 10 or higher to ensure that it can pass the
CTS.
If the device has SIM card slots, plug an activated SIM card into each slot.
If the device supports SMS, each SIM card must have its own number field
populated. For devices running Android 12 or
higher, all SIM cards must have support for storing abbreviated dialing
numbers (ADN). GSM and USIM cards with the telecom dedicated file
(DF
Telecom
) satisfy this requirement.
Developer UICC
To run CTS carrier API tests, the device needs to use a SIM with CTS carrier
privileges meeting the requirements specified in
Preparing the UICC
.
Android device configuration
Factory data reset the device:
Settings > Backup & reset > Factory data
reset
.
Set your device's language to English (
United States
):
Settings >
Language & input > Language
.
If the device supports customizing default fonts, set the default
sans-serif
font family to
Roboto
(the default
sans-serif
font family
used in AOSP builds).
Turn on the location setting if there's a GPS or Wi-Fi/cellular network
feature on the device:
Settings > Location > On
.
Connect to a Wi-Fi network that supports IPv6, can treat the DUT as an
isolated client
(see
Physical environment
above),
and has an internet connection:
Settings > Wi-Fi
.
Make sure that no lock pattern or password is set on the device:
Settings > Security > Screen lock > None
.
Enable
USB debugging
on your device:
Settings > Developer options >
USB debugging
.
Set the time to 12-hour format:
Settings > Date & time > Use 24-hour
format > Off
.
Set the device to stay awake:
Settings > Developer options > Stay Awake >
On
.
In
Android 5.x and 4.4.x only
, set the device to allow mock locations:
Settings > Developer options > Allow mock locations > On
.
In
Android 4.2 or higher
, turn off USB app verification:
Settings >
Developer options > Verify apps over USB > Off
.
In
Android 13 or higher
, set the device to allow mock modem:
Settings > Developer options > Allow Mock Modem > On
.
Launch the browser and dismiss any startup/setup screen.
Connect the desktop machine that will be used to test the device with a USB
cable.
Before running CTS, set Roboto2 as the sans-serif font using a user
accessible affordance (not hidden) setting.
File installation
Install and configure helper apps on the device.
Set up your device according to your CTS version:
CTS versions 2.1 R2 through 4.2 R4:
Set up your device (or emulator)
to run the accessibility tests with:
adb install -r
android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk
On the device, enable delegation:
Settings > Accessibility >
Accessibility > Delegating Accessibility Service
.
CTS versions 6.x or lower:
On devices that declare
android.software.device_admin
, set up your device to run the device
administration test using:
adb install -r android-cts/repository/testcases/CtsDeviceAdmin.apk`
In
Settings > Security > Select device administrators
, enable the
two
android.deviceadmin.cts.CtsDeviceAdminReceiver*
device
administrators. Ensure that
android.deviceadmin.cts.CtsDeviceAdminDeactivatedReceiver
and any
other preloaded device administrators remain disabled.
Copy the CTS media files to the device as follows:
- Navigate (
cd
) to the path where the media files are downloaded and
unzipped.
Change the file permissions:
chmod u+x copy_media.sh
Copy the necessary files:
To copy clips up to a resolution of 720x480, run:
./copy_media.sh 720x480
If you aren't sure of the maximum resolution, copy all of the files:
./copy_media.sh all
If there are multiple devices under adb, add the serial option
(
-s
) of a specific device to the end. For example, to copy up to
720x480 to the device with serial 1234567, run:
./copy_media.sh 720x480 -s 1234567