-
-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Lighthouse Flake]: No node found for selector: mat-option[ng-reflect-value="ADMIN"] #17279
Comments
@kevintab95 it looks like this flake might be related to the issue you hotfixed with #17661. The hotfix didn't fix it, but now more pages are failing due to a 403 error from donorbox |
Hi @U8NWXD, just to clarify, I think we'll need to have a "dev mode" flag for donorbox and not fetch from their domain when running e2e tests. The hotfix wasn't meant to fix a flake, the donation flow wasn't working as expected so we tried to fix that. |
Thanks! I understood that your hotfix wasn't about the flake--I just wanted to loop you in since it looked related. Any reason you prefer skipping donorbox during the tests rather than ignoring this particular error? I'm worried that if we cut donorbox out of the tests entirely, we'll miss bugs with it. Also /cc @seanlip since we're considering reducing test coverage |
I approve skipping donorbox during the tests -- I think there's a bit of an issue with exercising an external prod dependency multiple times a day due to our tests and on balance we probably shouldn't do it. (We've actually already missed bugs even with the tests anyway.) Will probably need to earmark this flow for manual testing on each release -- I will have a chat with @kevintab95 about that. |
OK, sounds good. Any suggestions on how to skip donorbox during the tests? Note that many tests run in prod mode, so that flag won't be enough |
I also agree with manual testing on each release. The tests have missed donorbox bugs, but if we don't test it at all, we should count on missing even more |
Hm, I'm not familiar enough with this part -- @kevintab95 might need to comment here. |
I think there are a couple of things to do here to fully solve this:
|
Dropping this flake to priority level since we haven't seen this flake since the beginning of April. |
This issue has been moved back to high priority since @462702985 has identified that the flake has been occurring at a high rate. Thanks for catching this @462702985 |
Hi -- just a note. This looks like it is due to console errors in the /donate page which are caused by an error in a third-party component (stripe). See https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1687048851060-5704.report.html and look at the accordion under "Browser errors were logged to the console". We should update our lighthouse config to try and exclude those errors. I saw GoogleChrome/lighthouse#9480 but I can't quite figure out how to modify .lighthouserc-1.js or .lighthouserc-base.js to include the ignoredPatterns option (my first attempt didn't work). I suggest trying to do this or just reducing the threshold for now so that PRs can go in. |
@seanlip I didn't find where we can add ignoredPatterns too. And I tried to add Content Security Policy but I cannot find http header of stripe.com. |
Note, this is causing consistent failures on Docker -- it needs to be investigated. |
Log:
Examples:
2023-02-08: https://github.com/oppia/oppia/actions/runs/4121545830/jobs/7117336965#step:8:6781
2023-02-22: https://github.com/oppia/oppia/actions/runs/4246546386/jobs/7383494361
2023-02-28: https://github.com/oppia/oppia/actions/runs/4289833043/jobs/7473254839#step:8:6815
2023-03-10: https://github.com/oppia/oppia/actions/runs/4387678295/jobs/7683259972#step:8:7056
2023-03-11: https://github.com/oppia/oppia/actions/runs/4390071617/jobs/7688233476#step:8:7054
Note that this log does not show the characteristic
mat-option[ng-reflect-value="ADMIN"]
error, but the uploaded lighthouse report does show the donorbox 403 error. In earlier examples only the donor page was affected, but now checks on all pages are failing, likely because of Fix #17608: Add donorbox widget.js script tag #17661.This flake hasn't been seen before, so it was probably introduced in the last few days. Here are the PRs merged over the last 5 days:
Despite the error in the output being
No node found for selector: mat-option[ng-reflect-value="ADMIN"]
, this appears to be related to donorbox. The Lighthouse reports say:2023-02-28:
2023-03-11:
This looks to be an issue with donorbox rate limiting us
The text was updated successfully, but these errors were encountered: