Axe - Web Accessibility Testing chrome extension slows down page load time

[LukaszJaro]

When using the chrome extension it adds up to 400ms+ in page load time

Environment:
Axe Version 4.0.0
Windows 10
Chrome Version 77.0.3865.90 (Official Build) (64-bit)

Expectation: To not have such a significant impact on speed.

Motivation: Deceiving page load times had me confused why pages were loading lightning fast in chrome incognito mode vs regular non-incognito mode, there should be a warning that there is a significant impact to page load time.

I'm also using service workers to create fully progressive web app where I normally get around 200ms page load time.

[straker]

We'll take a look to see what might be causing the slowdown.

[scurker]

@straker Lighthouse provides a good breakdown. I was running a local lighthouse audit against a static site with no javascript:

The 2 offending scripts from the extension are adapter.bundle.js and axe.min.js. I think these are changes that we need to make in the extension, possibly deferring loading of some things in our content script until the extension is opened.

However, @AutoSponge mentioned that axe Pro does require some listeners to always be there.

Related: #679

[dylanb]

@scurker @schne324 is there additional lazy-loading we could do to improve this?

[AutoSponge]

@scurker can we add this test to our repo? I'd like to start tracking perf benchmarks that impact the user, like this one. I think lighthouse has a cli now.

[scurker]

@dylanb tldr; Yes

We should defer dependencies that aren't needed until the extension is actually opened (e.g. axe-core, panel rendering, etc). @AutoSponge or @schne324 can correct me if I'm wrong on this, but some of axe-pro's events need to be available to talk to the backend so I don't think those can be lazy-loaded until later.

@AutoSponge I haven't figured out a way to get the lighthouse cli to load extensions. lighthouse <url> --chrome-flags="--load-extension=<path/to/extension>" seems to ignore that flag. exthouse is another possibility, but it expects a crx extension. So far the comparisons have been manual.

[AutoSponge]

@scurker I was able to run the check with our extension:

lighthouse http://google.com --chrome-flags="--disable-extensions-except=/Users/paulgrenier/Github/attest-browser-extensions/dist/axe/chrome, --load-extension=/Users/paulgrenier/Github/attest-browser-extensions/dist/axe/chrome" --view

• On google, our script took longer to execute than the host's script (751ms).

[scurker]

@AutoSponge Good catch! I had omitted the second flag, looks like we should be able to track performance related things with the following command:

lighthouse <url> \
  --chrome-flags="--load-extension=<path/to/extension> --disable-extensions-except=<path/to/extension>" \
  --only-categories=performance \
  --output=json > lighthouse.json

[erik-deque]

Deferring this until after 4.5.0 because it seems like a lot of work to do the lazy loading the way we want.

[dylanb]

@erik-deque can you check the status of this?

[erik-deque]

@scurker and I had a conversation about this particular item this week; we've improved the performance but we still have items we'd like to tackle to take it further.

[dylanb]

Extension version 4.18.2 improves performance by only evaluating axe-core in the page when the user presses the button, I am leaving this issue open because we still have some other planned improvements

[kospl]

@dylanb I'm using v 4.19.0 of extension for Chrome, and it still impacts every Performance audit in Lighthouse:

chrome-extension://lhdoppojpmngadmnindnejefpokejbdd/axe-versions/latest/axe.js is listed in Opportunities part of report. Repro steps:

1. Open webpage in Chrome
2. Open developer tools
3. Go to Lighthouse
4. Run Performance report
5. Check "Opportunities" - "Reduce unused JavaScript" section

[dylanb]

@kospl yes, that is why this ticket is still open, we made some improvements but still have some work to do