New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UI Tests for Simple Text Formatting #382

Merged
merged 10 commits into from Jun 19, 2017

Conversation

Projects
None yet
4 participants
@rachelmcr
Member

rachelmcr commented Jun 13, 2017

This PR adds a suite of UI tests for simple text formatting in the Aztec demo app, using the Android Espresso testing framework.

Before running the tests

Espresso advises disabling system animations on devices used for testing:

On your device, under Settings->Developer options disable the following 3 settings:

  • Window animation scale
  • Transition animation scale
  • Animator duration scale

One additional setup step is also required to handle an Espresso issue with clicks (see the caveats below):

On your device, under Settings -> Accessibility -> Touch & hold delay, set the delay to Long.

Test Details

Each test performs the following steps:

  1. Enter text (unformatted).
  2. Select a formatting button in the toolbar.
  3. Enter more text (with formatting).
  4. Verify the correct HTML appears in HTML mode for the unformatted and formatted text.

A couple caveats:

  • The Espresso framework sometimes performs a long click instead of a single click (see the Espresso code for an explanation). In the Aztec toolbar, this displays a hint instead of selecting the formatting button and causes the test to fail.
  • Espresso also has trouble with the editor's link dialog. Although it can identify the fields and buttons in the dialog (and TalkBack works with the dialog as expected) it clicks in the wrong position. This seems to be a bug with the coordinates select for the click action for a custom AlertDialog and so I reversed the link formatting test. (It enters the link HTML and then verifies that the link dialog displays the expected values.)

cc @catehstn @lancewillett @hoverduck

@rachelmcr

This comment has been minimized.

Show comment
Hide comment
@rachelmcr

rachelmcr Jun 13, 2017

Member

I just edited the original description to add a setup step for changing the "Touch & hold delay" setting to Long on the device used for testing. I'd forgotten this step earlier, but it should resolve most issues with the click turning into a long click in these tests (phones and tablets).

I do want to note that I observed a persistent problem with the click turning into a long click if the formatting button was selected as the first step in the test (before adding any text to the editor). That could indicate a potential performance issue when Aztec is first launched, and I went ahead and opened an issue to track that: #383

Member

rachelmcr commented Jun 13, 2017

I just edited the original description to add a setup step for changing the "Touch & hold delay" setting to Long on the device used for testing. I'd forgotten this step earlier, but it should resolve most issues with the click turning into a long click in these tests (phones and tablets).

I do want to note that I observed a persistent problem with the click turning into a long click if the formatting button was selected as the first step in the test (before adding any text to the editor). That could indicate a potential performance issue when Aztec is first launched, and I went ahead and opened an issue to track that: #383

@catehstn

This looks great!

Show outdated Hide outdated app/src/androidTest/java/org/wordpress/aztec/demo/BetterClickAction.java
@Override
@SuppressWarnings("unchecked")
public Matcher<View> getConstraints() {
Matcher<View> standardConstraint = isDisplayingAtLeast(65);

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

What's the 65 for? Does this relate to the comment at the top?

@catehstn

catehstn Jun 15, 2017

What's the 65 for? Does this relate to the comment at the top?

This comment has been minimized.

@rachelmcr

rachelmcr Jun 15, 2017

Member

Yes, the 65 here is a percentage: The default ClickAction in Espresso checks to make sure at least 90% of the element is displayed on-screen before clicking on it. The 65 here checks to make sure at least 65% of the element is displayed on-screen instead.

@rachelmcr

rachelmcr Jun 15, 2017

Member

Yes, the 65 here is a percentage: The default ClickAction in Espresso checks to make sure at least 90% of the element is displayed on-screen before clicking on it. The 65 here checks to make sure at least 65% of the element is displayed on-screen instead.

toggleHTMLView();
sourceText.check(matches(withText(unformattedText + "\n<pre>" + formattedText + "</pre>")));
}
}

This comment has been minimized.

@catehstn
@catehstn
public class TestUtils {
// Strings
public static String unformattedText = "hello";

This comment has been minimized.

@catehstn
@catehstn

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

Nice that we can have proper strings in Android - unlike iOS! :)

@catehstn

catehstn Jun 15, 2017

Nice that we can have proper strings in Android - unlike iOS! :)

// Check that HTML formatting tags were correctly added
toggleHTMLView();
sourceText.check(matches(withText(unformattedText + "\n<ol>\n\t<li>" + formattedText + "</li>\n</ol>")));

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

why is there a \t here?

@catehstn

catehstn Jun 15, 2017

why is there a \t here?

This comment has been minimized.

@rachelmcr

rachelmcr Jun 15, 2017

Member

The \t indents the list item on screen.

@rachelmcr

rachelmcr Jun 15, 2017

Member

The \t indents the list item on screen.

Show outdated Hide outdated ...androidTest/java/org/wordpress/aztec/demo/SimpleTextFormattingTests.java
}
/*
* This test is disabled because Espresso does not click in the correct position for the link dialog's OK button.

This comment has been minimized.

@catehstn

catehstn Jun 15, 2017

I think don't comment out a test if you've replaced it - instead probably add a comment inside the other one noting why it's tested that way and not this.

@catehstn

catehstn Jun 15, 2017

I think don't comment out a test if you've replaced it - instead probably add a comment inside the other one noting why it's tested that way and not this.

@catehstn

This comment has been minimized.

Show comment
Hide comment
@catehstn

catehstn Jun 15, 2017

@0nko ptal - if you're happy let's merge it.

catehstn commented Jun 15, 2017

@0nko ptal - if you're happy let's merge it.

@0nko 0nko self-assigned this Jun 15, 2017

@hoverduck

This comment has been minimized.

Show comment
Hide comment
@hoverduck

hoverduck Jun 16, 2017

@rachelmcr I finally got a chance to get my Android dev environment set back up and take a look. Everything checks out 👍 !

My only comment is that you should probably include an update to the README to provide the "Before running the tests" setup steps from the PR details. Is there maybe a way to automate those steps via a set of ADB commands prior to starting the tests?

hoverduck commented Jun 16, 2017

@rachelmcr I finally got a chance to get my Android dev environment set back up and take a look. Everything checks out 👍 !

My only comment is that you should probably include an update to the README to provide the "Before running the tests" setup steps from the PR details. Is there maybe a way to automate those steps via a set of ADB commands prior to starting the tests?

@0nko

This comment has been minimized.

Show comment
Hide comment
@0nko

0nko Jun 19, 2017

Contributor

After following the device configuration steps the tests pass. I'll add these to the README, like @hoverduck suggested.

Quickly searching for the adb commands I did find something for disabling the animations:

adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="window_animation_scale"'
adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="transition_animation_scale"'
adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="animator_duration_scale"'

And for setting a long tap delay:

adb shell settings put secure long_press_timeout <time in ms>

I haven't tested them, though. @rachelmcr, thank you so much for setting up the test environment and writing these!

Contributor

0nko commented Jun 19, 2017

After following the device configuration steps the tests pass. I'll add these to the README, like @hoverduck suggested.

Quickly searching for the adb commands I did find something for disabling the animations:

adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="window_animation_scale"'
adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="transition_animation_scale"'
adb shell content update --uri content://settings/system --bind value:s:0.0 --where 'name="animator_duration_scale"'

And for setting a long tap delay:

adb shell settings put secure long_press_timeout <time in ms>

I haven't tested them, though. @rachelmcr, thank you so much for setting up the test environment and writing these!

@0nko

This comment has been minimized.

Show comment
Hide comment
@0nko

0nko Jun 19, 2017

Contributor

:shipit:

Contributor

0nko commented Jun 19, 2017

:shipit:

@0nko 0nko merged commit a22b37b into develop Jun 19, 2017

3 checks passed

Buddybuild : AztecDemo/5800168c52aea90100a973ed Build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@0nko 0nko deleted the add/ui-text-formatting-tests branch Jun 19, 2017

@0nko 0nko modified the milestone: Open beta Jun 22, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment