[Dependency Updates] Test Instructions Documentations per Dependency Update #19552

ParaskP7 · 2023-11-07T12:23:05Z

Parent #17551
Internal Discussion pcdRpT-2Hp-p2

This PR adds a new documentations page with test instructions per dependency update.

FYI: The goal is to further automate the dependency update process by documenting all the specific dependency updates that require explicit testing instructions. The hypothesis here is that this approach would not only save time and effort for each update, but also elevate the quality of dependency update work by ensuring that the testing aspect is not left to chance.

PS: This PR is also slightly updating the pull request template by:

Splitting each section using a horizontal line, for grouping purposes (38a4d69),
Starting each section with a second-level heading, for consistency purposes (6b80fa7), and
Applying other minor changes, for consistency purposes (273a96f).

@irfano I added you as the main reviewer, not so randomly (you shared lots of good ideas), since I just wanted someone from the Jetpack/WordPress mobile team to be aware and sign-off on that change. Feel free to merge this PR directly yourself if you deem so.

To Test:

Proofread the new test instructions per dependency update documentation page.
- Purpose
- FAQ
- Content
- Smoke Test
- Special
Verify that each section is well written, can be easily understood and provide enough value while devs are working on a specific dependency update.

FYI: Note that the Smoke Test and Special sections can be further improved. However, I didn't want make this improvement now and instead added TODO for devs to keep improving the documentation themselves when and if needed.

Regression Notes

Potential unintended areas of impact
- N/A
What I did to test those areas of impact (or what existing automated tests I relied on)
- See To Test section above.
What automated tests I added (or what prevented me from doing so)
- N/A

PR submission checklist:

I have completed the Regression Notes.
I have considered adding accessibility improvements for my changes.
I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

UI Changes testing checklist:

Portrait and landscape orientations.
Light and dark modes.
Fonts: Larger, smaller and bold text.
High contrast.
Talkback.
Languages with large words or with letters/accents not frequently used in English.
Right-to-left languages. (Even if translation isn’t complete, formatting should still respect the right-to-left layout)
Large and small screen sizes. (Tablet and smaller phones)
Multi-tasking: Split screen and Pop-up view. (Android 10 or higher)

TODOs got added on both those test scenarios as I am not sure how to test this functionality and can't easily find the entry point to it. All I could do to test this was to manually comment out some code and verify that the functions were called after setting the device's date to days/weeks after today. Still, I think it is better to have that there and inform future dependency update testers about this functionality and the fact that this needs to be (somehow) tested, rather than ignore both completely. For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17892

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17960

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17936

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18552

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17994

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17996

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18554

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18113

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18090

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18102

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18076

TODOs got added on all those test scenarios. I think it is better to have that there and inform future dependency update testers about this functionality and the fact that this needs to be (somehow) tested, rather than ignore both completely. For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17571

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18060

TODOs got added on all those test scenarios. I think it is better to have that there and inform future dependency update testers about this functionality and the fact that this needs to be (somehow) tested, rather than ignore both completely.

FYI: Some smoke test instructions might have an 'Extra' section with addition instructions.

FYI: Every special test instructions have a 'Why & Extra' section to further explain why that dependency update is so special and how to deal with it.

peril-wordpress-mobile · 2023-11-07T12:23:11Z

	Warnings
⚠️	PR has more than 300 lines of code changing. Consider splitting into smaller PRs if possible.

Generated by 🚫 dangerJS

wpmobilebot · 2023-11-07T12:40:49Z

📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.

	App Name	WordPress
	Flavor	Jalapeno
	Build Type	Debug
	Version	pr19552-2f1403d
	Commit	`2f1403d`
	Direct Download	`wordpress-prototype-build-pr19552-2f1403d.apk`

Note: Google Login is not supported on these builds.

wpmobilebot · 2023-11-07T12:41:23Z

📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.

	App Name	Jetpack
	Flavor	Jalapeno
	Build Type	Debug
	Version	pr19552-2f1403d
	Commit	`2f1403d`
	Direct Download	`jetpack-prototype-build-pr19552-2f1403d.apk`

Note: Google Login is not supported on these builds.

irfano

Thank you for your efforts on this. I've added some comments for your review.

docs/test_instructions_per_dependency_update.md

This is done for 'WorkManager' and 'Preference', which were both incorrectly pointing to a specific line number, instead of the default line one. Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388166635

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388166635

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388214910

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388217235

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388224370

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388231514

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388239341

docs/test_instructions_per_dependency_update.md

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1389360965

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1389373449

wpmobilebot · 2023-11-10T13:35:34Z

	1 Warning
⚠️	This PR is larger than 300 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.

Generated by 🚫 Danger

irfano

Thank you for submitting this PR and addressing my comments.
LGTM. 👍🏻

ParaskP7 · 2023-11-10T15:41:22Z

Awesome, thank you so much reviewing and merging this, more so for being as thorough as you were, you rock! 🙇 ❤️ 🚀

ParaskP7 added 28 commits November 7, 2023 13:54

Docs: Add new test instructions per dependency update wiki page

05e8f59

Docs: Link test instructions per dependency update into pr template

e86d0b7

Docs: Split pr template per section using a horizontal line

38a4d69

Docs: Start each pr template section with a second-level heading

6b80fa7

Docs: Capitalize each pr template section

273a96f

Docs: Add todo bullet-points on regression notes pr template section

845623d

Docs: Add a faq section to test instructions per dep update wiki page

9dfce82

Docs: Add jp/wp test instruction prefix per test scenario

ae5e125

Docs: Remove unnecessary test scenario from the preference dependency

9875a57

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17960

Docs: Update test scenarios on the preference dependency

fd1248e

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17960

Docs: Add test scenarios for the exoplayer dependency

bae15b8

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17936

Docs: Add test scenarios for the firebase dependency

02dec46

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18552

Docs: Add test scenarios for the play services auth dependency

183af44

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17994

Docs: Add test scenarios for the play services code scanner dependency

65e29b1

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/17996

Docs: Add test scenarios for the navigation dependency

a5f9283

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18554

Docs: Add test scenarios for the navigation dependency

410597b

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18113

Docs: Add test scenarios for the zendesk dependency

b34962c

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18090

Docs: Add test scenarios for the jsoup dependency

bb0fe73

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18102

Docs: Add test scenarios for the auto service and kotlin poet deps

72db899

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18076

Docs: Add test scenarios for the ucrop dependency

f6a65e3

For more info see: https://github.com/wordpress-mobile/ WordPress-Android/pull/18060

Docs: Split dependency update test instructions using a horizontal line

4e135e6

Docs: Group content of dependency updates into logical units

4fcebb6

Docs: Move plugin group content of dependency updates on top

a3d0ad9

Docs: Add smoke test section

d474b57

FYI: Some smoke test instructions might have an 'Extra' section with addition instructions.

Docs: Add special section

4e5161f

FYI: Every special test instructions have a 'Why & Extra' section to further explain why that dependency update is so special and how to deal with it.

ParaskP7 added Tooling Tech Debt labels Nov 7, 2023

ParaskP7 added the dependency update label Nov 7, 2023

ParaskP7 added this to the Future milestone Nov 7, 2023

ParaskP7 requested a review from irfano November 7, 2023 12:23

ParaskP7 mentioned this pull request Nov 7, 2023

Dependency Updates #17551

Closed

50 tasks

irfano requested changes Nov 9, 2023

View reviewed changes

ParaskP7 added 7 commits November 10, 2023 12:27

Docs: Remove unnecessary line numbers from links and point to file

847d28b

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388166635

Docs: Add missing white spaces on play services auth test instructions

c243351

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388214910

Docs: Add info message about external contributor on play services auth

2de223d

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388217235

Docs: Change login wording to log-in on play services code scanner

84bf847

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388224370

Docs: Add an extra testing step on zendesk

b1dd15b

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388231514

Docs: Add an extra info about disabling/enabling block editor on jsoup

f3870ce

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1388239341

irfano reviewed Nov 10, 2023

View reviewed changes

docs/test_instructions_per_dependency_update.md Outdated Show resolved Hide resolved

ParaskP7 added 2 commits November 10, 2023 15:28

Docs: Remove remaining line numbers from links and point to file

1079693

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1389360965

Docs: Split preference into androidx and android testing instructions

2f1403d

Related PR Comment: https://github.com/wordpress-mobile/ WordPress-Android/pull/19552#discussion_r1389373449

irfano approved these changes Nov 10, 2023

View reviewed changes

irfano merged commit 9fc79ce into trunk Nov 10, 2023
20 checks passed

irfano deleted the docs/test_instructions_per_dependency_update branch November 10, 2023 15:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Dependency Updates] Test Instructions Documentations per Dependency Update #19552

[Dependency Updates] Test Instructions Documentations per Dependency Update #19552

ParaskP7 commented Nov 7, 2023

peril-wordpress-mobile bot commented Nov 7, 2023

wpmobilebot commented Nov 7, 2023 •

edited

wpmobilebot commented Nov 7, 2023 •

edited

irfano left a comment

wpmobilebot commented Nov 10, 2023

irfano left a comment

ParaskP7 commented Nov 10, 2023

[Dependency Updates] Test Instructions Documentations per Dependency Update #19552

[Dependency Updates] Test Instructions Documentations per Dependency Update #19552

Conversation

ParaskP7 commented Nov 7, 2023

To Test:

Regression Notes

peril-wordpress-mobile bot commented Nov 7, 2023

wpmobilebot commented Nov 7, 2023 • edited

wpmobilebot commented Nov 7, 2023 • edited

irfano left a comment

Choose a reason for hiding this comment

wpmobilebot commented Nov 10, 2023

irfano left a comment

Choose a reason for hiding this comment

ParaskP7 commented Nov 10, 2023

wpmobilebot commented Nov 7, 2023 •

edited

wpmobilebot commented Nov 7, 2023 •

edited