Tips, Tricks and Troubleshooting

Maestro Studio

Video showcasing Maestro Studio

Use Maestro Studio to instantly discover the exact commands needed to interact with your app.

Maestro Studio automatically generates examples of how you can interact with the selected element in your Flows. You can either double click on the example to have it be executed directly or copy it, read the docs and other things via the hotkeys available.

Running Javascript

While it is possible to run Javascript from within a Maestro test, it's fairly limited in what it can do.

Maestro uses a subset of Javascript called RHINO, which is needs to be able to execute it from within the Java environment in which it runs. As a result, there are some limitations to what your JS scripts can do:

• They have to be written in Javascript .js and not Typescript .ts.
• The JS implementation is limited to ES15 features.
• It can not import/export to and from other files, so all logic must be contained within the single file.
• Has to use the HTTP API implemented by Maestro rather than fetch or any other native network implementation.

info

Maestro currently has optional ES2022 support using GraalJS. This feature is still experimental, but is intended to be default in the future. You can use this in your tests now, but please be mindful. To find out more, see the Maestro docs here.

It's important to go over what's in the docs to understand what you can and can't do.

Maestro Known Issues

There is one major issue you will run into when running Maestro tests on iOS.

Interacting with nested components on iOS

In some cases, you may run into issues with nested tappable / accessible elements on iOS. You can resolve these issues by enabling accessibility for the inner component and disabling it for the outer container.

We have 2 exported consts you can use here.

• isAccessible and
• iosE2ENonAccessibleProps

If you need the SUBMIT button to be identified my Maestro, you would do something like:

<TouchableOpacity {...iosE2ENonAccessibleProps} testID={'wrapper-button-not-needed'} >
  <Text>WRAPPER</Text>
  <TouchableOpacity accessible={isAccessible} testID={'submit-button-id'}>
    <Text>SUBMIT</Text>
  </TouchableOpacity>
</TouchableOpacity>

This will ensure that we can find the correct id to tapOn in the test. You can read up more about it here in Maestro docs.

You can find a list of additional known issues here from the Maestro docs.

Formatting Your Code

You will find there can be many steps within your tests. For the sake of everybody's time, please use whitespace and appropriate comments throughout the steps of your test. This will make it easier for others to read and understand what is happening in the test, making it easier to debug.

For an example, please see this test: e2e/tests/group-mode/open-groups-carousel-joining.yaml.

Common Maestro Commands

You will find a list of all the commands you can use to drive your tests in the docs here from Maestro  Commands List from Maestros

Below are some examples but please click the link above for more detail.

tapOn

- tapOn:
    id: "id" # or any other selector

assertVisible

- assertVisible:
    # Same exact parameters as in tapOn or any other command that uses selectors

Selectors

- tapOn: # or any other command that works with selectors
    text: "Text"     # (optional) Finds text or accessibility text that matches regexp. 
    id: "id"         # (optional) Finds id that matches regexp
    index: 0         # (optional) 0-based index of the view to select among those that match all other criteria
    point: 50%,50%.  # (optional) Relative position on screen. 50%,50%: Middle of screen
    point: 50, 50    # (optional) Exact coordinates on screen, x:50 y:50, in pixels.
    width: 100       # (optional) Finds element of a given width
    height: 100      # (optional) Finds element of a given height
    tolerance: 10    # (optional) Tolerance to apply when comparing width and height
    enabled: true    # (optional) Searches for view with a given "enabled" state
    checked: true    # (optional) Searches for view with a given "checked" state
    focused: true    # (optional) Searches for view with a given "focused" state
    selected: true   # (optional) Searches for view with a given "selected" state
    optional: false  # (default: false) If set to true, test won't fail if view can't be found

scrollUntilVisible

- scrollUntilVisible:
    element:
      id: "viewId" # or any other selector
    direction: DOWN # DOWN|UP|LEFT|RIGHT (optional, default: DOWN)
    timeout: 50000 # (optional, default: 20000ms)
    speed: 40 # 0-100 (optional, default: 40) Scroll speed. Higher values scroll faster.
    visibilityPercentage: 100 # 0-100 (optional, default 100) Percentage of element visible in viewport

Have a look at the Maestro documents which contains example flows

Linting

Maestro has the ability to access environment variables both in .yaml and .js files. However, when access these environment variables in the Maestro .js files, they will throw an <variable> is not defined lint warning. To avoid this, you can add the following comment to the top of your .js file:

/* global <variable> */

This will ensure we do not contribute to any additional linting errors we have in our codebase.

Addtionally, we have added these variables below to the list of globally defined Maestro variables, which skips linting.

  globals: {
    output: 'writable',
    BRAND: 'readonly',
    http: 'readonly',
    maestro: 'readonly',
  },

Reviewing Work

• Review your own work before asking others to. Chances are you'll find issues with your work faster than anyone else, and save others time reviewing your code.

Here are some criteria to check before a test can be considered complete

Criteria	Notes
Test runs across both platforms (Unless scenario is specific to one platform)	Both IOS and Android needs to be covered
Test runs across both brands (Unless scenario is specific to one brand)	Both Neds and Ladbrokes (and Tab NZ) need to be covered
Test can be run consecutively without requiring manual set up.	This means all set up is implemented within the test-setup files. Eg: Using API to set up data or using mocked requests to set up the required test data.
No duplicate code has been created during the creation of the test	Avoid creating a duplicate flow if it already exists from another test
Maestro test yaml includes an ID linking back to the Test case	For example the Jira ticket
All test steps in test case are covered by the test script	Always cross check the original test case to assure every step is covered. This should be confirmed by a QA also.

Our Scripts

• For a list of our available scripts, see RN's package.json.

Debugging Tests

We want our tests to be reliable and as flake free as possible, and the only way to identify flake is to run the test and check that it doesn't fail. As you're writing your test, run it often! Tests can take a long time to complete depending on what you're trying to cover, (STILL RELEVANT TO MAESTRO)

so if you aren't actively using the simulator during the time of writing, run your tests and ensure that they are passing regularly: if not, debug why and fix. A failing test is great once, as it shows that there is a problem, but once we've identified that the problem exists through the first failure (either on our or the app's end), you should always be taking steps to remedy it. If the failure is a result of an actual issue, follow up with the relevant team to ensure that it's tracked; if it's flake with your test, adjust your test.