Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing: Add publish tests using the block editor #11696

Merged
merged 5 commits into from May 21, 2019

Conversation

rachelmcr
Copy link
Member

This PR adds a new Gutenberg test suite with two publish flow tests using the block editor. These are the same publish tests we already had running for Aztec (which are now in a single Aztec test suite).

This PR also switches the main navigation test to use the block editor instead of Aztec. (The navigation itself is the same, but we have to define which editor we expect to open after tapping the editor button in the bottom nav bar.)

Note: For now, the block editor tests only run in English. We'll need to look at either adding test IDs to Gutenberg or making the localizable strings from Gutenberg accessible to these tests (less stable) to run them in other device languages.

To test:

  • Pull the latest mobile secrets, or create a file with test credentials at ~/.mobile-secrets/iOS/WPiOS/WPUITestCredentials.swift (based on the template at /WordPressUITests/WPUITestCredentials-Template.swift).
  • Open the simulator and confirm the device language is set to Ennglish.
  • Run the all test suites except LoginTests in the WordPressUITests target.

Update release notes:

  • If there are user facing changes, I have added an item to RELEASE-NOTES.txt.

@rachelmcr rachelmcr added the Testing Unit and UI Tests and Tooling label May 15, 2019
@rachelmcr rachelmcr added this to the 12.5 milestone May 15, 2019
@rachelmcr rachelmcr self-assigned this May 15, 2019
@rachelmcr rachelmcr added this to In progress in E2E testing ramp up via automation May 15, 2019
@rachelmcr rachelmcr added Testing Unit and UI Tests and Tooling and removed Testing Unit and UI Tests and Tooling labels May 15, 2019
@rachelmcr rachelmcr requested a review from JavonDavis May 15, 2019 14:00
E2E testing ramp up automation moved this from In progress to Needs review May 17, 2019
Copy link
Contributor

@JavonDavis JavonDavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code LGTM! and the tests work pretty well! 👍🏽 I just have a small nitpick and a case I think we should cover..

There's something I noticed when I was initially testing this out, running the Gutenberg tests before the Aztec tests led to a failure as a result of the block editor still being on... I cleaned the build folder and went again and now I noticed that each time the app was being reinstalled so this couldn't be reproduced anymore(weird but I don't think that's the expected behavior). To reproduce this a simple change of EditorGutenbergTests to EditorAGutenbergTests can do the trick... I know in the current scenario unless the tests are ran separately this shouldn't happen or but I think this can also occur if the app isn't re-installed and the BlockEditor is still on.. I'm thinking we can just always ensure the block editor is off for the AztecEditor test in a similar approach to what's being done in the BlockEditor test...

The nitpick I had is that we're using

        super.setUp()
        // In UI tests it is usually best to stop immediately when a failure occurs.
        continueAfterFailure = false

        let app = XCUIApplication()
        app.launchArguments = ["NoAnimations"]
        app.activate()

        // Media permissions alert handler
        systemAlertHandler(alertTitle: "“WordPress” Would Like to Access Your Photos", alertButton: "OK")

in the setUp for most of the tests and probably will use it in future tests as well, any chance we could reuse that block of code in a function?

@rachelmcr
Copy link
Member Author

I'm thinking we can just always ensure the block editor is off for the AztecEditor test in a similar approach to what's being done in the BlockEditor test...

Good call, that's an easy thing to add. 👍

The nitpick I had is that we're using [...] in the setUp for most of the tests and probably will use it in future tests as well, any chance we could reuse that block of code in a function?

Yeah, I hadn't thought about that. I'll take a look at what makes the most sense there for reusing all or most of that block of code. Thanks!

@peril-wordpress-mobile
Copy link

Warnings
⚠️ PR has more than 500 lines of code changing. Consider splitting into smaller PRs if possible.

Generated by 🚫 dangerJS

@rachelmcr rachelmcr requested a review from JavonDavis May 20, 2019 10:10
Copy link
Contributor

@JavonDavis JavonDavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚢

E2E testing ramp up automation moved this from Needs review to Reviewer approved May 20, 2019
@jkmassel jkmassel modified the milestones: 12.5, 12.6 May 20, 2019
@rachelmcr rachelmcr merged commit 41da008 into develop May 21, 2019
E2E testing ramp up automation moved this from Reviewer approved to Done May 21, 2019
@rachelmcr rachelmcr deleted the add/publish-test-with-block-editor branch May 21, 2019 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing Unit and UI Tests and Tooling
Projects
Development

Successfully merging this pull request may close these issues.

None yet

4 participants