Acts Automation Framework For Android

1. Introduction

The Android Comms Test Suite, is a lightweight Python-based automation tool set that is used to
perform automated testing of current and upcoming Android devices. It provides a simple execution
interface; a set of pluggable libraries for accessing commercially available devices, Android devices, and
a collection of utility functions to further ease test development. It is an ideal desktop tool for a wireless
stack developer or integrator whether exercising a new code path, performing sanity testing, or running
extended regression test suites.
Included in the tests/google directory are a bundle of tests, many of which can be run with as little as one
or two Android devices with wifi, cellular, or bluetooth connectivity, including:
1. Wifi tests for access point interopability, enterprise server integration, WiFi scanning, WiFi
auto join, and round trip time.
2. Bluetooth tests for low energy, GATT, SPP, and bonding.
3. Cellular tests for circuit switch and IMS calling, data connectivity, messaging, network
switching, and WiFi hotspot.

2. Prerequisites

Details about system dependencies, setup and installation guide for installing ACTS automation
framework are also present in README file in <root>/tools/test/connectivity/acts/README.md
1. Linux or Mac machine (Windows is supported but not well tested)
2. Adb
3. Fastboot
4. Python 3.4+ (preferred) or Python2.7
5. python3-setuptools
6. Install methods
7. sudo apt-get install python3-setuptools
8. pip3 install python3-setuptools
9. Usb cables
10. User debug or Engineering build image flashed on Android devices under test.

3. Getting started

There is an in depth guide on how to setup your environment and get ACTS (Android CommsTest
Suite) setup (README.md as mentioned above). Following sections will simplify ACTS setup for
Bluetooth Automation testing.

3.1 Installing SL4A :

Assumption: you already have a lunch target selected and built.1. Connect the android devices you want to run automation against.
2. Navigate to <root>/external/sl4a
3. Run this command: mm -j8
4. Depending on the branch you checked out, it will either install on all the android devices
connected or just create the apk. If for some reason it is an older version that just build the apk,
the location of the app is: <root>/out/target/product/<target>/data/app/sl4a/sl4a.
5. cd to above directory
6. Install the app:
7. To install on pre M branches: $ adb install -r sl4a.apk
8. To install on M+ devices: $ adb install -r -g sl4a.apk

3.2 Installing the framework :

Navigate to <root>/tools/test/connectivity/acts/framework
Run: sudo python3.4 setup.py develop
If this command is successful then you should be able to enter the command: act.py
Entering it should spew usage information that looks something like this:

3.3 Setting up your config file :

Config files tell the framework what Android devices are in use, where the logpath will be, and where
the testcase path will be. There is a sample.config.json file in the framework directory.
Note: The majority of testcases require two devices so the below config file has two serial numbers of
two separate Android devices. It is advisable to keep all the configs file under home directory, eg :
~/configs

Eg : CarKit_Phone.config file:

{ "_description": "This is an example skeleton test configuration file.",
"testbed":
[
{
"_description": "A testbed with two android devices.",
"name": "carkit_phone",
"AndroidDevice": [{"serial":"Carkit serial"}, {"serial":"Phone serial"}]
}
],
"logpath": "/tmp/logs",
"testpaths": ["../tests"],
"example_custom_config_param": "custom_param"
}

Note : To get started quickly, just change out the “AndroidDevice” list of serial numbers with the
devices you have. Logs will be located at “logpath” + testbed name. In this case logs will be here after
each test: /tmp/logs A soft link is created at each test run to easily access the latest run: /tmp/logs/latest/,
The “testpaths” param is set to a relative path. Which means you have to run all your tests from the
framework directory. You can make this absolute if you want. If you don’t want bugreports on failures
add the custom param below testpaths: “bugreport_on_fail” : “False”.

3.4 Running your first test :

A very simple test that only requires 1 Android device is BtAirplaneModeTest, this tests various
airplane mode scenarios and how it affects Bluetooth state. Here is how to run it:
$ act.py -c ~/config/single_device_airplane_mode.config -tc BtAirplaneModeTest
-c param == config file
-tc param == test class to run

Specify a specific testcase within the test file with a colon.

$ act.py -c ~/config/single_device_airplane_mode.config –tc

BtAirplaneModeTest:test_bt_on_toggle_airplane_mode_on
Run a suite of tests with -tf.

$ act.py -c ~/config/single_device_airplane_mode.config –tf ./test_suite.txt

BtAirplaneModeTest:test_bt_on_toggle_airplane_mode_on
BtKillProcessTest
All test files contain is a list of test classes to run and/or a list of testcases to run.

3.5 Useful Custom Params

 1. For Bluetooth tests, by default we take a bugreport on failure. If you don’t want this to
Happen, add this custom param in your config file:
        a. "no_bug_report_on_fail": "True"
2. For Bluetooth tests that inherit from BluetoothBaseTestClass, if you want tests to rerun
Once before determining failure provide this custom param:
        a. "bt_auto_rerun": "True"

3.6 Bluetooth Tests Classes

Test Classes

Test Class Name
BleScanApiTest
BleAdvertiseApiTest
GattApiTest
UniqueFilteringTest
AdvertisingSetTest
Bt5ScanTest
PhyTest
FilteringTest
BleStressTest
GattConnectTest
GattReadTest
GattWriteTest
GattNotifyTest
BeaconSwarmTest
ConcurrentBleAdvertisementDiscoveryTest
ConcurrentBleAdvertisingTest
ConcurrentBleScanTest
BtStressTest
BtBasicFunctionalityTest
BleOpportunisticScanTest
RfcommTest
RfcommStressTest
RfcommLongevityTest

GattOverBrEdrTest
BleOnLostOnFoundTest
BleOnLostOnFoundStressTest
BleBackgroundScanTest
BluetoothPanTest
BtAirplaneModeTest
GattLongevityTest
BtPreFlightTest
BtKillProcessTest
BtFactoryResetTest
BtFactoryResetTest
HidDeviceTest

All of these test suites (and more) are located at:
acts/tests/google/ble
acts/tests/google/bt
Each of these can be executed as described in section “Running your first test”.
For example:

act.py -c <config> -tc FilteringTest

3.7 A2DP and AVRCP tests

For A2DP and AVRCP testcases media needs to be pushed to a device. Provide a path in the
config file a folder you would like to push to the test phone. Note that you should have at least 2
music files that are supported by your Android device (preferably mp3 format). The custom
param should be named “local_media_path”. Here is an example of one in use (in bold).

{ "_description": "This is an example skeleton test configuration file.",
"testbed":
[
{
"_description": "A testbed for Android Auto Testing",
"name": "auto_test",
"AndroidDevice": ["<carkit_serial>", "<phone_serial>"]
}
],
"logpath": "/tmp",
"testpaths": ["<path_to_tests>"],
" local_media_path ": "<path_to_directory_with_music>"
}

3.8 PTS / Iphone Cmd Line Tool

The PTS (Profile Tuning Suite) Command Line tool if for controlling a single (soon multiple)
Android device for passing PTS tests or to connect to a non android device. Setting it up and running it
is pretty simple. The code is already in AOSP here: tools/test/connectivity/acts/tests/google/bt/pts. The
only change in the config file you need is to add the “target_bt_addr_address” which corresponds to
the device you want to test against. In this example it will be your PTS dongle / IOS device. See below
config file for the example. 

Suggested config setup:

{ "_description": "This is an example skeleton test configuration file.",
"testbed":
[
{
"_description": "A testbed for Android Auto Testing",
"name": "pts_test",
"AndroidDevice": ["<carkit_serial>"]
}
],
"logpath": "/tmp",
"testpaths": ["<path_to_tests>"],
"no_bug_report_on_fail": "True",
" target_mac_address ": "<pts_bt_address / Iphone BTAddress>”
}

Once you setup your config file, kick off the cmd line as such:
$act.py -c <config> -tc BtCmdLineTest

By pressing TAB twice you will see the entire list of commands at your disposal: 

This tool supports auto-complete. For example is you start typing “gatts” (short or GATT Server)
and tap tab twice you will get a list of available commands for the GATT Server.