Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor transition checker action tests to be less brittle #1919

Merged
merged 1 commit into from Feb 5, 2020

Conversation

@huwd
Copy link
Contributor

huwd commented Feb 5, 2020

Trello: https://trello.com/c/XT3ekLSK/442-rewrite-checker-tests-to-make-them-less-brittle

Why?

On the 30th and 31st we needed to bulk update a range of actions at once. After doing so we discovered some the actions that had been removed were required for tests. This made the process more intensive.

We can have higher degrees of confidence that some actions will remain for the duration of the transition period. The actions with the highest degree of confidence would make good candidates for tests, as they should weather any future changes to actions in the checker.

Those actions are suggested to be:

  • Business: Getting a customs agent
  • Individual: EU Settled status

Also, occasionally when we add new actions they will also match the criteria these tests select for. This will cause new additions to the test's result page, which can lead to changes in order the results are rendered. The data-track-action is a tracking attribute which explicitly records that, current tests with has_css? searching for the attribute key and value return only found/not found.

A better strategy would be to return the string that is found and check that against our expectations, updating them if necessary.

What's changed?

  • Changes results page tests to use T099 (customs agent) as test candidate for business questions/results tests
  • Changes results page tests to use S001 (EU settled status) as test candidate for business questions/results tests
  • Changes integration test question selections to ensure that the above actions do / do not appear depending on the test
  • Refactor the test for data-track-action into two tests, one for the existance of a govuk link with the correct href and a data-track-action. Then a separate test that does a string match against the expected content of that string.
Specifically, switch to using actions we have a higher degree of confdience will remain constants across the next year, also separate the checker for data-track-action out into a separate expects, which will give better expectation mismatch responses if we add new actions that change the position of these tested actions on the results page
@huwd huwd requested review from laurentqro, koetsier and vanitabarrett Feb 5, 2020
@huwd huwd self-assigned this Feb 5, 2020
@bevanloon bevanloon temporarily deployed to finder-front-less-britt-wjxt2j Feb 5, 2020 Inactive
@huwd huwd merged commit 1f9d705 into master Feb 5, 2020
3 checks passed
3 checks passed
continuous-integration/jenkins/branch This commit looks good
Details
continuous-integration/jenkins/publishing-e2e-tests Publishing end-to-end tests succeeded on Jenkins
Details
continuous-integration/jenkins/security No security issues found
Details
@huwd huwd deleted the less-brittle-tests branch Feb 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.