-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(custom-scheduler): stabilize custom scheduler testing #16533
test(custom-scheduler): stabilize custom scheduler testing #16533
Conversation
(note, this passes over and over again on my fork (4 in a row now, running 5th) - example - https://github.com/mikehardy/Anki-Android/actions/runs/9341737197/job/25709045233 ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rejecting due to whitespace changes in commit 1 on ReviewerTest
- the entire file is shown as being recreated
Approve after this is fixed, suggestion is a nitpick for clarity
Yeah the spacing change is ridiculous and will definitely be changed if this passes muster at all:
Seems like it might - with nitpick for clarity 👍 - so will clean this up a bit and repush |
without this, `./gradlew jacocoAndroidTestReport` would not work on the command line, but the tests would work in Android Studio, which is quite bizarre
with appropriate retries it appears to work reliably
462cd03
to
1384973
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fantastic, cheers!
Succeeded first time here as well, and the merge queue will run it again and then it'll run again again on push to main, so 3 more runs to give it a final look and hope it is actually non-flaky If this starts to flake again - for anyone reading this in the future - just readd the Flaky annotations, do not change any of the other work here as it was all necessary (even the retries!) in order for it to work locally even. |
Purpose / Description
After implementing the reviewer fragment thing in the new reviewer with #16455 the custom scheduler test was failing for me on the command line so my customary / frequent
./gradlew jacocoTestReport
did not passI believe others did not notice it because:
1- it is not executed in CI because of a Flaky annotation
2- I suppose people developing locally either don't run tests or run them from Android Studio? I use the command line
I need that to pass, so I looked into it, and this or something like this will fix it
Fixes
Approach
1- realize by view hierarchy inspection that the hierarchies are different from the command line and from Android Studio. I have no idea. They are though. So...allow either name for the the layout flip or ease 3 button view resource to match 🤷
2- realize that timing issues are still present (even for me locally) but if we use the otherwise-unused retry rule with a retry of 3, it works every time locally! So I am passing now
3- try to get this working in CI so it doesn't start silently failing again and sending me on a yak shave. Bumping retry to 10 seems to do it? 🙏
Note there was a line-ending issue in initial push that has been peeled off to #16536
How Has This Been Tested?
It's all tests, it is made of tests. But this was a flake, so it has a a higher burden of proof. I'm 9 for 9 running this so I think I actually stabilized it
Learning (optional, can help others)
Checklist
Please, go through these checks before submitting the PR.