Run Espresso tests

Step 1: Prepare an App

You can build your Application .apk file using IDE or in the CLI.

After that, the appropriate file can be found in build/outputs/apk/<profile> directory.

Additional information can be found here.

Step 2: Prepare a Test Suite app

It’s necessary to run assembleAndroidTest configuration to create a .apk file with Test Suite.

You can do it in the IDE or in the CLI.

Here is an example of IDE configuration:

Here is an Example of CLI execution:

After that, the appropriate file can be found in app/build/outputs/apk/androidTest/<profile> directory.

Step 3: Upload the App and the Test Suite

Upload your Application and Test Suites apps to Mobitru using our REST API.

Retrieve and store the id of every uploaded .apk, because it will be used to trigger an Execution.

Here is an example cURL request to upload the app:

curl --location --request POST 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/spaces/artifacts' \
--header 'x-File-Name: <app_name>.apk' \
--header 'X-Content-Type: application/zip' \
--header 'Authorization: Bearer <ACCESS_KEY>' \
--form 'alias="apk_debug"' \
--form 'file=@"/path/to/app/file/app.apk"' \
--form 'checksum="None"'

Below you can find a sample of the response with an id of the uploaded app:

{
    "id": "d32173ea-ceaf-457b-9fdc-ef7882c0d1ca",
    "name": "6f6a9dbf-5a88-4e22-8c9a-70930e94428f.apk",
    "realName": "<app_name>.apk",
    "bid": "7a21f4ad-e322-43e2-9649-0209de6a9752",
    "wid": "0",
    "private": true,
    "uploadedBy": "<user_name>@test.com",
    "uploadedAt": 1672165595,
    "verified": false,
    "target": "android",
    "contentType": "application/zip",
    "contentLength": 0,
    "href": "quarantine/6f6a9dbf-5a88-4e22-8c9a-70930e94428f.apk",
    "checksum": "None",
    "alias": "<app_name>",
    "apk": null,
    "ipa": null
}

Step 4: Find and take device

Find and take appropriate device using our APIs (not needed if shards number more than 1).

Here is an example of cURL request to find a device via API :

curl --location --request GET 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/device/android?model=Pixel 6' \
--header 'Authorization: Bearer <ACCESS_KEY>'

Below you can find a sample of the response with capabilities for found devices:

[
    {
        "desiredCapabilities": {
            "platformName": "Android",
            "platformVersion": "12",
            "deviceName": "GOOGLE Pixel 6a",
            "udid": "26281JEGR04493"
        }
    },
    {
        "desiredCapabilities": {
            "platformName": "Android",
            "platformVersion": "12",
            "deviceName": "GOOGLE Pixel 6",
            "udid": "19161FDF600DJP"
        }
    },
    {
        "desiredCapabilities": {
            "platformName": "Android",
            "platformVersion": "13",
            "deviceName": "GOOGLE Pixel 6 Pro",
            "udid": "1A281FDEE007T3"
        }
    }
]

Here is an example of cURL request to take a device via API:

curl --location --request POST 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/device/26281JEGR04493' \
--header 'Authorization: Bearer <ACCESS_KEY>'

Below you can find a sample of the response with capabilities for the device:

{ 
    "desiredCapabilities": {
        "platformName": "Android",
        "platformVersion": "12",
        "deviceName": "GOOGLE Pixel 6a",
        "udid": "26281JEGR04493"
    }
}

Step 5: Run tests

After taking a Device (in case of a single shard), you can run your Espresso tests using all existing information.

Here is an example of cURL request to run all tests using a single shard:

curl --location --request POST 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/tests/espresso' \
--header 'Authorization: Bearer <ACCESS_KEY>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "serial": "26281JEGR04493",
    "app_artifact_id": "<app_ipa_artifact_id>",
    "test_artifact_id": "<tests_ipa_artifact_id>",
    "runner_class":"androidx.test.runner.AndroidJUnitRunner"
}'

Below you can find a sample of the response with the Run ID:

{
    "id": "df5d5bb4-d43c-4964-8ddd-ecf4f6d67b71",
    "engine": "espresso",
    "status": "pending",
    "config": {
        "device_spec": {
            "serial": "26281JEGR04493",
            "platform": "android"
        },
        "app_artifact_id": "<APP_IPA_ARTIFACT_ID>",
        "test_artifact_id": "<TESTS_IPA_ARTIFACT_ID>",
        "shard_number": 1,
        "duration": 60
    },
    "summary": "",
    "artifacts": {},
    "created_at": "yyyy-mm-ddT19:16:41.598897Z",
    "updated_at": "yyyy-mm-ddT19:16:41.598897Z"
}

If necessary, you can cancel pending execution by performing the separate API:

Here is an example of cURL request to cancel the Execution:

curl --location --request PUT 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/tests/espresso/<RUN_ID>/cancel' \
--header 'Authorization: Bearer <ACCESS_KEY>'

As result, you will receive 204 No Content response.

Filter test cases

By default, all tests from your Test Suite will be executed, but it’s possible to use multiple options to filter test cases for execution:

ParameterDescriptionValues
tests_regexRegex value for filtering tests by name*.testNamePart.*
classfull name of a Class with tests.
will be executed only these tests		com.example.app.tests.espresso.Basic.ChangeTextBehaviorTest
notClassfull name of a Class with tests.
will be executed all tests except those located in the Class		com.example.app.tests.espresso.Basic.ChangeTextBehaviorTest
packagefull name of a Package with tests.
will be executed only for tests located in the Package		com.example.app.tests.espresso.Basic.Other
notPackagefull name of a Package with tests.
will be executed all tests except those located in the Package		com.example.app.tests.espresso.Basic.Other
annotationfull name of an Annotation, which is used for marking tests.
will be executed only annotated tests		com.example.app.tests.espresso.Basic.SampleAnnotation
notAnnotationfull name of an Annotation, which is used for marking tests.
will be executed all tests except annotated		com.example.app.tests.espresso.Basic.SampleAnnotation
sizekeywords, which related to test size and appropriate test annotations @SmallTest, @MediumTest, @LargeTestsmall, large, or medium

Here is an example of cURL request for different cases:

curl --location --request POST 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/tests/espresso' \
--header 'Authorization: Bearer <ACCESS_KEY>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "serial": "26281JEGR04493",
    "app_artifact_id": "<app_ipa_artifact_id>",
    "test_artifact_id": "<tests_ipa_artifact_id>",
    "runner_class":"androidx.test.runner.AndroidJUnitRunner",
    "test_filters": [
        "tests_regex .*same.*",
        "class com.example.app.tests.espresso.Basic.ChangeTextBehaviorTest",
        "notClass com.example.app.tests.espresso.Basic.ChangeTextBehaviorKtTest",
        "package com.example.app.tests.espresso.Basic",
        "notPackage com.example.app.tests.espresso.Basic.Other",
        "annotation com.example.android.testing.espresso.BasicSample.Other.SampleAnnotation",
        "notAnnotation com.example.android.testing.espresso.BasicSample.Other.SampleOtherAnnotation",
        "size large"
    ]
}'

Use custom instrumentation arguments

By default, no specific instrumentation arguments are used in an execution configuration, but it’s possible to provide custom parameters by adding a particular field in the API request body.
The Field format is “<key>[space]<value>”.

Here is an example of cURL request with custom argument:

curl --location 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/tests/espresso' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <ACCESS_KEY>' \
--data '{
    "serial": "26281JEGR04493",
    "app_artifact_id": "<app_ipa_artifact_id>",
    "test_artifact_id": "<tests_ipa_artifact_id>",
    "runner_class":"androidx.test.runner.AndroidJUnitRunner",
    "instrument_options": ["log true"]
}'

Speed up tests execution

By default, all test cases in the Suite will be executed sequentially. According to this, it can take a long time to execute a lot of tests. So, instead of running the Suite sequentially, you can split test cases into different groups called shards. Each shard will be executed in parallel on a separate device. Also, it’s possible to use the Tests Filtering together with sharding.

To enable test sharding in a Run, you need to specify device filter info and the shards number in the API request body.

Here is an Example of device filters:

    "device_spec": {
        "name": "Samsung" //full or partial name for filtering devices
    },
    
    "device_spec": {
        "platform_version": "11" //devices version
    },

Here is an Example of cURL request to execute tests with multiple sharding enabled:

curl --location --request POST 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/tests/espresso' \
--header 'Authorization: Bearer <ACCESS_KEY>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "device_spec": {
        "name": "Samsung",
        "platform_version": "11"
    },
    "shard_number": 2,
    "app_artifact_id": "<app_ipa_artifact_id>",
    "test_artifact_id": "<tests_ipa_artifact_id>",
    "runner_class":"androidx.test.runner.AndroidJUnitRunner"
}'

Important note: you don’t need to take devices in case of multiple shards, because Mobitru will do this automatically, based on device_spec, which is located in the Request Body.

Step 6: Get the Run details

After the finish of the Execution, you can get the Run details via API.

Here is an example of cURL a request to get such details:

curl --location --request GET 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/tests/espresso/<RUN_ID>' \
--header 'Authorization: Bearer <ACCESS_KEY>'

Below you can find a sample of the response with artifacts.log ID:

{
    "id": "<RUN_ID>",
    "engine": "espresso",
    "status": "completed",
    "config": {
        "device_spec": {
            "serial": "26281JEGR04493",
            "platform": "android"
        },
        "app_artifact_id": "<APP_IPA_ARTIFACT_ID>",
        "test_artifact_id": "<TESTS_IPA_ARTIFACT_ID>",
        "shard_number": 1,
        "duration": 60
    },
    "summary": "Shard 0 - 26281JEGR04493: Test run completed",
    "artifacts": {
        "log-0": "01efd72d-d70c-409e-9b5a-a61019b082f7"
    },
    "created_at": "yyyy-mm-ddT19:16:41.598897Z",
    "updated_at": "yyyy-mm-ddT19:16:41.598897Z"
}

In case of multiple shards, a separate log will be prepared for every device:

{
    "id": "<RUN_ID>",
    "engine": "espresso",
    "status": "completed",
    "config": {
        "device_spec": {
            "name": "Samsung",
            "platform": "android",
            "platform_name": "11"
        },
        "app_artifact_id": "<APP_IPA_ARTIFACT_ID>",
        "test_artifact_id": "<TESTS_IPA_ARTIFACT_ID>",
        "shard_number": 2,
        "duration": 60
    },
    "summary": "Shard 1 - <DEVICE1_UDID>: Test run completed; Shard 0 - <DEVICE2_UDID>: Test run completed",
    "artifacts": {
        "log-0": "b0c5fc37-c4f8-487a-a7b1-488a63a811b7",
        "log-1": "75ae6324-263b-4d23-9b58-fc9fe562fd20"
    },
    "created_at": "yyyy-mm-ddT19:16:41.598897Z",
    "updated_at": "yyyy-mm-ddT19:16:41.598897Z"
}

Each log item will contain details for tests, which were executed on particular device.

Step 7: Get execution log

Also, you can get a base log via a separate API.

Here is an example of cURL request to get such log:

curl --location --request GET 'https://app.mobitru.com/billing/unit/<BILLING_UNIT>/automation/api/v1/spaces/artifacts/<LOG_ID>/download' \
--header 'Authorization: Bearer <ACCESS_KEY>'

Below you can find an example of a response with the log content:

INSTRUMENTATION_STATUS: class=com.example.android.testing.espresso.BasicSample.ChangeTextBehaviorTest
INSTRUMENTATION_STATUS: current=1
INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
INSTRUMENTATION_STATUS: numtests=10
INSTRUMENTATION_STATUS: stream=
.....
.....
.....
INSTRUMENTATION_STATUS: current=10
INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
INSTRUMENTATION_STATUS: numtests=10
INSTRUMENTATION_STATUS: stream=.
INSTRUMENTATION_STATUS: test=retrieveNumbers_textWithDigits
INSTRUMENTATION_STATUS_CODE: 0
INSTRUMENTATION_RESULT: stream=

Time: 16.816

OK (10 tests)


INSTRUMENTATION_CODE: -1
Scroll to Top