Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UI Tests for Simple Text Formatting #382

Merged
merged 10 commits into from Jun 19, 2017
Merged

Conversation

@rachelmcr
Copy link
Member

rachelmcr commented Jun 13, 2017

This PR adds a suite of UI tests for simple text formatting in the Aztec demo app, using the Android Espresso testing framework.

Before running the tests

Espresso advises disabling system animations on devices used for testing:

On your device, under Settings->Developer options disable the following 3 settings:

  • Window animation scale
  • Transition animation scale
  • Animator duration scale

One additional setup step is also required to handle an Espresso issue with clicks (see the caveats below):

On your device, under Settings -> Accessibility -> Touch & hold delay, set the delay to Long.

Test Details

Each test performs the following steps:

  1. Enter text (unformatted).
  2. Select a formatting button in the toolbar.
  3. Enter more text (with formatting).
  4. Verify the correct HTML appears in HTML mode for the unformatted and formatted text.

A couple caveats:

  • The Espresso framework sometimes performs a long click instead of a single click (see the Espresso code for an explanation). In the Aztec toolbar, this displays a hint instead of selecting the formatting button and causes the test to fail.
  • Espresso also has trouble with the editor's link dialog. Although it can identify the fields and buttons in the dialog (and TalkBack works with the dialog as expected) it clicks in the wrong position. This seems to be a bug with the coordinates select for the click action for a custom AlertDialog and so I reversed the link formatting test. (It enters the link HTML and then verifies that the link dialog displays the expected values.)

cc @catehstn @lancewillett @hoverduck

@rachelmcr
Copy link
Member Author

rachelmcr commented Jun 13, 2017

I just edited the original description to add a setup step for changing the "Touch & hold delay" setting to Long on the device used for testing. I'd forgotten this step earlier, but it should resolve most issues with the click turning into a long click in these tests (phones and tablets).

I do want to note that I observed a persistent problem with the click turning into a long click if the formatting button was selected as the first step in the test (before adding any text to the editor). That could indicate a potential performance issue when Aztec is first launched, and I went ahead and opened an issue to track that: #383

Copy link

catehstn left a comment

This looks great!

@Override
@SuppressWarnings("unchecked")
public Matcher<View> getConstraints() {
Matcher<View> standardConstraint = isDisplayingAtLeast(65);

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

What's the 65 for? Does this relate to the comment at the top?

This comment has been minimized.

@rachelmcr

rachelmcr Jun 15, 2017 Author Member

Yes, the 65 here is a percentage: The default ClickAction in Espresso checks to make sure at least 90% of the element is displayed on-screen before clicking on it. The 65 here checks to make sure at least 65% of the element is displayed on-screen instead.

toggleHTMLView();
sourceText.check(matches(withText(unformattedText + "\n<pre>" + formattedText + "</pre>")));
}
}

This comment has been minimized.

public class TestUtils {

// Strings
public static String unformattedText = "hello";

This comment has been minimized.

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

Nice that we can have proper strings in Android - unlike iOS! :)


// Check that HTML formatting tags were correctly added
toggleHTMLView();
sourceText.check(matches(withText(unformattedText + "\n<ol>\n\t<li>" + formattedText + "</li>\n</ol>")));

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

why is there a \t here?

This comment has been minimized.

@rachelmcr

rachelmcr Jun 15, 2017 Author Member

The \t indents the list item on screen.

}

/*
* This test is disabled because Espresso does not click in the correct position for the link dialog's OK button.

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

I think don't comment out a test if you've replaced it - instead probably add a comment inside the other one noting why it's tested that way and not this.

@catehstn
Copy link

catehstn commented Jun 15, 2017

@0nko ptal - if you're happy let's merge it.

@0nko 0nko self-assigned this Jun 15, 2017
@hoverduck
Copy link

hoverduck commented Jun 16, 2017

@rachelmcr I finally got a chance to get my Android dev environment set back up and take a look. Everything checks out 👍 !

My only comment is that you should probably include an update to the README to provide the "Before running the tests" setup steps from the PR details. Is there maybe a way to automate those steps via a set of ADB commands prior to starting the tests?

@0nko
Copy link
Contributor

0nko commented Jun 19, 2017

After following the device configuration steps the tests pass. I'll add these to the README, like @hoverduck suggested.

Quickly searching for the adb commands I did find something for disabling the animations:

adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="window_animation_scale"'
adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="transition_animation_scale"'
adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="animator_duration_scale"'

And for setting a long tap delay:

adb shell settings put secure long_press_timeout <time in ms>

I haven't tested them, though. @rachelmcr, thank you so much for setting up the test environment and writing these!

@0nko
Copy link
Contributor

0nko commented Jun 19, 2017

:shipit:

@0nko 0nko merged commit a22b37b into develop Jun 19, 2017
3 checks passed
3 checks passed
Buddybuild : AztecDemo/5800168c52aea90100a973ed Build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@0nko 0nko deleted the add/ui-text-formatting-tests branch Jun 19, 2017
@0nko 0nko modified the milestone: Open beta Jun 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.