Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate identity verification accessibility tests to improve test speed #10359

Merged
merged 1 commit into from
Apr 4, 2024

Conversation

mitchellhenke
Copy link
Contributor

@mitchellhenke mitchellhenke commented Apr 3, 2024

🛠 Summary of changes

I did some testing to ensure that the same pages are tested for accessibility, this should only be a reduction in duplicative tests. This is intended to be a speed improvement on #4000 and the intervening changes since then.

I added a small patch to keep track of which pages the test file was running accessibility checks on to be more sure we aren't losing accessibility coverage. We'd expect to see some duplication since the mobile test shares many of the same pages. The results don't suggest that we've lost any coverage, so this change should be a reduction in time spent without any reduction in test coverage.

Before:

# ~45 seconds locally
[["/verify/agreement", 2], ["/verify/cancel", 1], ["/verify/document_capture", 2], ["/verify/enter_password", 3], ["/verify/how_to_verify", 1], ["/verify/hybrid_handoff", 1], ["/verify/personal_key", 3], ["/verify/phone", 3], ["/verify/ssn", 2], ["/verify/verify_info", 2], ["/verify/welcome", 3]]

After:

# ~25 seconds locally
[["/verify/agreement", 2], ["/verify/cancel", 1], ["/verify/document_capture", 2], ["/verify/enter_password", 2], ["/verify/how_to_verify", 1], ["/verify/hybrid_handoff", 1], ["/verify/personal_key", 2], ["/verify/phone", 2], ["/verify/ssn", 2], ["/verify/verify_info", 2], ["/verify/welcome", 3]]
Patch
 require 'rails_helper'
 require 'axe-rspec'
+ACCESSIBLE_PAGES = {}

 RSpec.feature 'Accessibility on IDV pages', :js, allowed_extra_analytics: [:*] do
   describe 'IDV pages' do
@@ -9,6 +10,10 @@ RSpec.feature 'Accessibility on IDV pages', :js, allowed_extra_analytics: [:*] d
       create(:service_provider, :active, :in_person_proofing_enabled)
     end

+    after(:all) do
+      puts ACCESSIBLE_PAGES.sort.inspect
+    end
+
     scenario 'how to verify page' do
       allow(IdentityConfig.store).to receive(:in_person_proofing_enabled).and_return(true)
       allow(IdentityConfig.store).to receive(:in_person_proofing_opt_in_enabled).and_return(true)
diff --git a/spec/support/matchers/accessibility.rb b/spec/support/matchers/accessibility.rb
index b9e295ecc6..379183baad 100644
--- a/spec/support/matchers/accessibility.rb
+++ b/spec/support/matchers/accessibility.rb
@@ -310,6 +310,9 @@ class AccessibleName
 end

 def expect_page_to_have_no_accessibility_violations(page, validate_markup: true)
+  ACCESSIBLE_PAGES[page.current_path] ||= 0
+  ACCESSIBLE_PAGES[page.current_path] += 1
+
   expect(page).to be_axe_clean.according_to(
     :section508, :"best-practice",
     :wcag21aa
--

… speed

changelog: Internal, Testing, Consolidate identity verification accessibility tests to improve test speed
Copy link
Contributor

@zachmargolis zachmargolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

big win!


expect_page_to_have_no_accessibility_violations(page)
end

scenario 'how to verify page' do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we consolidate this one in as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gets kind of difficult since it's not in the default happy path. Enabling in-person-proofing adds a screen in the middle. We could probably modify the existing step helpers to accommodate it though (especially since it will be enabled more widely soon).

I'm going to merge this for now though.

Comment on lines 37 to 38
fill_in t('idv.form.password'), with: Features::SessionHelper::VALID_PASSWORD
click_continue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we're missing accessibility checks for a lot of these screens. I'd wonder if we could throw it in here for the password step?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it'd probably be useful to have some expectation of which pages have been checked to make sure we're covering everything we expect to, but I'd leave that to another PR?

@mitchellhenke mitchellhenke merged commit 8d02c7f into main Apr 4, 2024
2 checks passed
@mitchellhenke mitchellhenke deleted the mitchellhenke/faster-idv-accessibility-specs branch April 4, 2024 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants