| <html devsite> |
| <head> |
| <title>Compatibility Test Suite</title> |
| <meta name="project_path" value="/_project.yaml" /> |
| <meta name="book_path" value="/_book.yaml" /> |
| </head> |
| <body> |
| <!-- |
| Copyright 2017 The Android Open Source Project |
| |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| |
| |
| <h2 id="how-does-the-cts-work">How does the CTS work?</h2> |
| |
| <p>The Compatibility Test Suite (CTS) is a free, commercial-grade test suite, |
| available for <a href="/compatibility/cts/downloads">download</a>. The CTS represents the |
| "mechanism" of compatibility.</p> |
| |
| <p>The CTS runs on a desktop machine and executes test cases directly on |
| attached devices or an emulator. The CTS is a set of unit tests designed to be |
| integrated into the daily workflow (such as via a continuous build system) of |
| the engineers building a device. Its intent is to reveal incompatibilities |
| early on, and ensure that the software remains compatible throughout the |
| development process.</p> |
| |
| |
| <p>The CTS is an automated testing harness that includes two major software components:</p> |
| <ul> |
| <li> |
| The CTS tradefed test harness runs on your desktop machine and manages test execution. |
| </li> |
| <li> |
| Individual test cases are executed on the Device Under Test (DUT). The test |
| cases are written in Java as JUnit tests and packaged as |
| Android .apk files to run on the actual device target. |
| </li> |
| </ul> |
| |
| <p>The Compatibility Test Suite Verifier (CTS Verifier) is a supplement to the |
| CTS available for <a href="/compatibility/cts/downloads">download</a>. CTS Verifier |
| provides tests for APIs and functions that cannot be tested on a stationary |
| device without manual input (e.g. audio quality, accelerometer, etc).</p> |
| |
| <p>The CTS Verifier is a tool for manual testing and includes the following software components:</p> |
| <ul> |
| <li> |
| <p>The CTS verifier app that is executed on the DUT and collects the results.<p> |
| </li> |
| <li> |
| <p>The executable(s) or script(s) that are executed on the desktop machine to |
| provide data or additional control for some test cases in the CTS Verifier |
| app.</p> |
| </li> |
| </ul> |
| |
| <h2 id="workflow">Workflow</h2> |
| |
| <div class="attempt-right" style="width:383px"> |
| <img src="/compatibility/cts/images/cts-0.png" alt="CTS flow" height="340px" id="figure1" /> |
| <p class="img-caption"> |
| <strong>Figure 1.</strong> How to use CTS |
| </p> |
| </div> |
| |
| <p>This diagram summarizes CTS workflow. Please refer to the subpages of this |
| section starting with <a href="/compatibility/cts/setup">Setup</a> for detailed |
| instructions.</p> |
| |
| <h2 id="types-of-test-cases">Types of test cases</h2> |
| <p>The CTS includes the following types of test cases:</p> |
| <ul> |
| <li> |
| <em>Unit tests</em> test atomic units of code within the Android platform; |
| e.g. a single class, such as java.util.HashMap. |
| </li> |
| <li> |
| <p><em>Functional tests</em> test a combination of APIs together in a higher-level use-case.</p> |
| </li> |
| </ul> |
| Future versions of the CTS will include the following types of test cases: |
| <ul> |
| <li> |
| <em>Robustness tests</em> test the durability of the system under stress. |
| </li> |
| <li> |
| <em>Performance tests</em> test the performance of the system against defined benchmarks, |
| for example rendering frames per second. |
| </li> |
| </ul> |
| <h2 id="areas-covered">Areas covered</h2> |
| <p>The unit test cases cover the following areas to ensure compatibility:</p> |
| <table> |
| <thead> |
| <tr> |
| <th>Area</th> |
| <th>Description</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>Signature tests</td> |
| <td>For each Android release, there are XML files describing the signatures of all |
| public APIs contained in the release. The CTS contains a utility to check those API |
| signatures against the APIs available on the device. The results from signature |
| checking are recorded in the test result XML file.</td> |
| </tr> |
| <tr> |
| <td>Platform API Tests</td> |
| <td>Test the platform (core libraries and Android Application Framework) APIs as documented |
| in the SDK <a href="https://developer.android.com/reference/classes">Class Index</a> to |
| ensure API correctness, including correct class, attribute and method signatures, |
| correct method behavior, and negative tests to ensure expected behavior for |
| incorrect parameter handling.</td> |
| </tr> |
| <tr> |
| <td>Dalvik Tests</td> |
| <td>The tests focus on testing the Dalvik Executable Format.</td> |
| </tr> |
| <tr> |
| <td>Platform Data Model</td> |
| <td>The CTS tests the core platform data model as exposed to application developers |
| through content providers, as documented in the |
| SDK <a href="https://developer.android.com/reference/android/provider/package-summary"> |
| android.provider</a> package: contacts, browser, settings, etc.</td> |
| </tr> |
| <tr> |
| <td>Platform Intents</td> |
| <td>The CTS tests the core platform intents, as documented in the |
| SDK <a href="https://developer.android.com/guide/appendix/g-app-intents"> |
| Available Intents</a>.</td> |
| </tr> |
| <tr> |
| <td>Platform Permissions</td> |
| <td>The CTS tests the core platform permissions, as documented in the |
| SDK <a href="https://developer.android.com/reference/android/Manifest.permission"> |
| Available Permissions</a>.</td> |
| </tr> |
| <tr> |
| <td>Platform Resources</td> |
| <td>The CTS tests for correct handling of the core platform resource types, |
| as documented in the |
| SDK <a href="https://developer.android.com/guide/topics/resources/available-resources"> |
| Available Resource Types</a>. This includes tests for: simple values, drawables, nine-patch, |
| animations, layouts, styles and themes, and loading alternate resources.</td> |
| </tr> |
| </tbody> |
| </table> |
| |
| </body> |
| </html> |