Compatibility Test Suite (CTS)
is a free, commercial-grade test suite used by
device manufacturers to help ensure that their devices are compatible, and to report test results
for validations. CTS is intended to be integrated into the daily workflow (such as through a
continuous build system) of the engineers building a device. CTS's intent is to reveal
incompatibilities early on, and ensure that the software remains compatible throughout the
development process.
CTS is available for
download
as a binary or
as source in the
Android Open Source
Project (AOSP)
. CTS runs on a desktop machine and executes test cases directly on
attached devices or an emulator.
CTS contains two major software components:
-
The
Trade Federation
test harness runs on
your desktop machine and manages test execution. Trade Federation offers the ability to
shard tests
across multiple Devices Under Test (DUTs). You may also use the
suite retry
feature to retry the failures only rather than the full suites, greatly reducing re-run time.
-
The individual test cases are executed on the DUT. The test
cases are written in Java as JUnit tests and packaged
Android .apk files to run on the actual device target.
Compatibility Test Suite Verifier (CTS Verifier)
The
Compatibility Test Suite Verifier (CTS Verifier)
is a supplement to
CTS available for
download
. The CTS Verifier
provides tests for APIs and functions that cannot be tested on a stationary
device without manual input (for example, audio quality and accelerometer) or manual positioning.
The CTS Verifier is a tool for manual testing and includes the following software components:
-
The CTS Verifier app that is executed on the DUT (and on a 2nd device for distance
measurement requirements) and collects the results.
-
The executable(s) or script(s) that are executed on the desktop machine to
provide data or additional control for some test cases in the CTS Verifier
app.
Workflow
Figure 1.
How to use CTS
This diagram summarizes CTS workflow. Please refer to the subpages of this
section starting with
Setup
for detailed
instructions.
Types of test cases
CTS includes the following types of test cases:
Unit test coverage
The unit test cases cover the following areas to ensure compatibility:
Area
|
Description
|
Signature tests
|
For each Android release, there are XML files describing the signatures of all
public APIs contained in the release. The CTS contains a utility to check those API
signatures against the APIs available on the device. The results from signature
checking are recorded in the test result XML file.
|
Platform API Tests
|
Test the platform (core libraries and Android Application Framework) APIs as documented
in the SDK
Class Index
to
ensure API correctness, including correct class, attribute and method signatures,
correct method behavior, and negative tests to ensure expected behavior for
incorrect parameter handling.
|
Dalvik Tests
|
The tests focus on testing the Dalvik Executable Format.
|
Platform Data Model
|
The CTS tests the core platform data model as exposed to application developers
through content providers, as documented in the
SDK
android.provider
package: contacts, browser, settings, etc.
|
Platform Intents
|
The CTS tests the core platform intents, as documented in the
SDK
Available Intents
.
|
Platform Permissions
|
The CTS tests the core platform permissions, as documented in the
SDK
Available Permissions
.
|
Platform Resources
|
The CTS tests for correct handling of the core platform resource types,
as documented in the
SDK
Available Resource Types
. This includes tests for: simple values, drawables, nine-patch,
animations, layouts, styles and themes, and loading alternate resources.
|