Acceptance test doc update #297

rahat2134 · 2024-03-28T17:41:22Z

Fix #292 : This pr updates the wiki of accpetance test for mobile.

rahat2134 · 2024-03-28T17:43:35Z

jnvtnguyen

Made a partial review.

jnvtnguyen · 2024-03-30T04:07:06Z

Acceptance-Tests.md

+
+In this example, the `discardCurrentChanges()` function checks if the viewport width corresponds to a mobile device, and if so, clicks on the mobile-specific discard button. Otherwise, it clicks on the desktop-specific discard button. Finally, it confirms the discard action. This approach allows us to maintain a single set of tests while accommodating differences between desktop and mobile environments.
+
+### console errors logging functionality in Acceptance Tests


Move this section (console errors section) to above since it can be used in all cases (mobile/desktop), also should be capitalized beginning word and rest non capitalized. Console errors...

done, Thanks :)

jnvtnguyen · 2024-03-30T04:07:23Z

Acceptance-Tests.md

+
+### console errors logging functionality in Acceptance Tests
+
+Now, Acceptance Tests are capable of finding console errors during CUJ's and it will break the test. But in some cases where the console error can be ignored and not in high priority right now then we can ignore that with help of `consoleReporter.setConsoleErrorsToIgnore`


consoleReporter -> ConsoleReporter

Done, thanks :)

jnvtnguyen · 2024-03-30T04:08:51Z

Acceptance-Tests.md

+
+Now, Acceptance Tests are capable of finding console errors during CUJ's and it will break the test. But in some cases where the console error can be ignored and not in high priority right now then we can ignore that with help of `consoleReporter.setConsoleErrorsToIgnore`
+```typescript
+ConsoleReporter.setConsoleErrorsToIgnore([


This is only used for situations where the console errors are acceptable and they should be more precise as the console errors only show up as a byproduct of the test (e.g. ConsoleReporter.setConsoleErrorsToIgnore([
'Blog Post with the given title exists already. Please use a different title.',
]);) <- This only shows up as an error since the test errors it. The error in this case isn't generic like status of 500 but the actual error.

Done, Thanks :)

jnvtnguyen · 2024-03-30T04:10:08Z

Acceptance-Tests.md

+Now, Acceptance Tests are capable of finding console errors during CUJ's and it will break the test. But in some cases where the console error can be ignored and not in high priority right now then we can ignore that with help of `consoleReporter.setConsoleErrorsToIgnore`
+```typescript
+ConsoleReporter.setConsoleErrorsToIgnore([
+  /Failed to load resource: the server responded with a status of 500/,


We should add another statement on how to add errors that are already existing or issues that are bugs. To ignore these, we should put them in the actual console-reporter.ts utility (should probably mention this as a file in the tree above as well). You can look in there to see how we do it but basically we put the error regex into a constant array and put a TODO to the filed issue.

Done, Thanks :)

Acceptance-Tests.md

jnvtnguyen · 2024-04-03T04:14:29Z

Acceptance-Tests.md

@@ -31,7 +31,8 @@ oppia/core/tests/
    │  ├── puppeteer-utils.ts
    │  ├── show-message-utils.ts
    │  ├── test-constants.ts
-    │  └── user-factory.ts
+    │  ├── user-factory.ts
+    │  └── console-reporter.ts


Add an explanation for this file (console-reporter.ts) inside, The directory structure is as follows... The puppeteer-testing-utilities... new bullet point.

Done, Thanks

Co-authored-by: Justin Nguyen <70992422+jnvtnguyen@users.noreply.github.com>

…loper-docs into Acceptance-Test-doc-update

…2134/oppia-web-developer-docs into Acceptance-Test-doc-update

rahat2134 · 2024-04-15T14:02:54Z

@jnvtnguyen PTAL.

rahat2134 · 2024-04-18T13:03:28Z

@jnvtnguyen PTAL.

jnvtnguyen

Sorry for the delay.

jnvtnguyen · 2024-04-18T18:39:53Z

Acceptance-Tests.md

+    'https://pencilcode.net/lib/pencilcodeembed.js - Failed to ' +
+      'load resource: net::ERR_CERT_DATE_INVALID'
+  ),
+];


Maybe add a section for CONSOLE_ERRORS_TO_FIX as well and how you should add error regexes here when you come by an error which is a bug (prompt developer to file an issue (or if already existing do not) and add a TODO). Though this will be gone once we completely remove all console error bugs.

jnvtnguyen · 2024-04-18T18:41:36Z

Acceptance-Tests.md

+
+We can handle these differences by including conditional statements in the `utils` file, using the `isViewportAtMobileWidth()` function to execute commands specific to mobile devices.
+
+For Eg.


For example:

…2134/oppia-web-developer-docs into Acceptance-Test-doc-update

rahat2134 · 2024-04-22T16:22:42Z

@jnvtnguyen PTAL.

jnvtnguyen · 2024-04-22T17:39:47Z

Acceptance-Tests.md

+  ),
+];
+```
+To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which have the details of the issue. If the error is new, also raise it in the issue section.


Suggested change

To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which have the details of the issue. If the error is new, also raise it in the issue section.

To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which points to the existing issue number. If the error doesn't have any corresponding issue, then file a new issue on our [issue tracker](https://github.com/oppia/oppia/issues).

done @jnvtnguyen PTAL.

jnvtnguyen

One small thing, other than that LGTM

jnvtnguyen · 2024-04-22T19:07:17Z

Acceptance-Tests.md

+
+Similar to desktop, we also have acceptance tests for mobile to ensure responsiveness and uninterrupted user journeys on small screen devices. While the tests themselves remain largely the same for both desktop and mobile, there are some differences. For instance, large full menus on desktop may be converted to dropdowns, hamburger menus, or other shortcuts on mobile, requiring additional actions to complete the tests.
+
+### How to Write Tests for Mobile


How to write tests for mobile

@jnvtnguyen Done.

rahat2134 · 2024-04-22T19:12:04Z

@seanlip PTAL. Its approved by @jnvtnguyen. Merge this if no further review is required.

Ash-2k3 · 2024-04-24T12:02:13Z

Thanks @jnvtnguyen for the review!!! Assigning to @seanlip for final pass, Thanks!

seanlip

Thanks @rahat2134 (and @jnvtnguyen)! Overall this looks quite good. I just have a few small grammar/readability points, PTAL.

seanlip · 2024-04-24T12:31:57Z

Acceptance-Tests.md

@@ -126,6 +128,114 @@ make run_tests.acceptance suite="blog-editor-tests/check-blog-editor-unable-to-p

 10) The test must be thoroughly tested before submitting a PR. The test can be run locally by running the following command as mentioned above or you can run the test on the CI server by pushing your code to the remote branch in your fork. The CI server will run the test and will show the result.

+### Console errors logging functionality in Acceptance Tests
+
+Acceptance Tests have the capability to detect console errors during CUJ's, potentially resulting in test failures. However, there are scenarios where certain console errors can be deemed acceptable and should not cause the test to fail. In order to ignore certain errors like these, you can use `ConsoleReporter.setConsoleErrorsToIgnore`, which takes in an array of error regexes to match the errors that can be acceptable. For instance, errors like `Blog Post with the given title exists already. Please use a different title.`, which occurs during the 'blog-editor-tests/try-to-publish-a-duplicate-blog-post-and-get-blocked' test, is ignored using the ConsoleReporter since it is an acceptable error in the context of the test. When passing acceptable errors like these to the ConsoleReporter, you should be specific and not use vague errors like `Failed to load resource...`, below is an example of this usage:


ignore certain errors --> ignore errors

errors like --> an error like

Split "Below is an example ..." at the end into its own sentence.

seanlip · 2024-04-24T12:32:09Z

Acceptance-Tests.md

+]);
+```
+
+To handle errors that need to be ignored and are are not specific to any acceptance test, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_IGNORE` array and explain with a comment why this error should be ignored.


Duplicate "are".

seanlip · 2024-04-24T12:33:16Z

Acceptance-Tests.md

+```
+To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which points to the existing issue number. If the error doesn't have any corresponding issue, then file a new issue on our [issue tracker](https://github.com/oppia/oppia/issues).
+
+Note: This will be removed after the bug is resolved.


It's unclear what the note and what "this" refer to here. I suggest inlining it with the paragraph above (if you're talking about the issue) ... or clarifying if you're talking about something else.

rahat2134 · 2024-04-24T12:57:30Z

@seanlip PTAL.

seanlip

Thanks, couple more things.

seanlip · 2024-04-24T13:11:07Z

Acceptance-Tests.md

-To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which points to the existing issue number. If the error doesn't have any corresponding issue, then file a new issue on our [issue tracker](https://github.com/oppia/oppia/issues).
-
-Note: This will be removed after the bug is resolved.
+To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which points to the existing issue number(This will be removed after the bug is resolved). If the error doesn't have any corresponding issue, then file a new issue on our [issue tracker](https://github.com/oppia/oppia/issues).


... add a TODO comment which points to the existing issue number (this comment should be removed when the bug is resolved).

seanlip · 2024-04-24T13:11:14Z

Acceptance-Tests.md

@@ -130,14 +130,16 @@ make run_tests.acceptance suite="blog-editor-tests/check-blog-editor-unable-to-p

 ### Console errors logging functionality in Acceptance Tests

-Acceptance Tests have the capability to detect console errors during CUJ's, potentially resulting in test failures. However, there are scenarios where certain console errors can be deemed acceptable and should not cause the test to fail. In order to ignore certain errors like these, you can use `ConsoleReporter.setConsoleErrorsToIgnore`, which takes in an array of error regexes to match the errors that can be acceptable. For instance, errors like `Blog Post with the given title exists already. Please use a different title.`, which occurs during the 'blog-editor-tests/try-to-publish-a-duplicate-blog-post-and-get-blocked' test, is ignored using the ConsoleReporter since it is an acceptable error in the context of the test. When passing acceptable errors like these to the ConsoleReporter, you should be specific and not use vague errors like `Failed to load resource...`, below is an example of this usage:
+Acceptance Tests have the capability to detect console errors during CUJ's, potentially resulting in test failures. However, there are scenarios where certain console errors can be deemed acceptable and should not cause the test to fail. In order to ignore errors like these, you can use `ConsoleReporter.setConsoleErrorsToIgnore`, which takes in an array of error regexes to match the errors that can be acceptable. For instance, an error like `Blog Post with the given title exists already. Please use a different title.`, which occurs during the 'blog-editor-tests/try-to-publish-a-duplicate-blog-post-and-get-blocked' test, is ignored using the ConsoleReporter since it is an acceptable error in the context of the test. When passing acceptable errors like these to the ConsoleReporter, you should be specific and not use vague errors like `Failed to load resource...`.


CUJ's --> CUJs

seanlip

LGTM, thanks @rahat2134 !

rahat2134 added 7 commits March 20, 2024 23:19

acceptance test doc update

a83c866

changes dones

86959c5

changes dones

71c56e7

changes dones

8b1782e

changes dones

4c1254e

update wiki of acceptance tests for mobile acceptance tests

b504102

resolve merge conflict

0d7ecb4

Ash-2k3 assigned jnvtnguyen Mar 28, 2024

jnvtnguyen requested changes Mar 30, 2024

View reviewed changes

jnvtnguyen assigned rahat2134 and unassigned jnvtnguyen Mar 30, 2024

changes done

e2ce082

rahat2134 requested a review from jnvtnguyen March 31, 2024 02:45

changes done

677f8c9

Ash-2k3 assigned jnvtnguyen and unassigned rahat2134 Mar 31, 2024

jnvtnguyen requested changes Apr 3, 2024

View reviewed changes

jnvtnguyen assigned rahat2134 and unassigned jnvtnguyen Apr 3, 2024

rahat2134 and others added 9 commits April 3, 2024 17:49

Update Acceptance-Tests.md

37b71de

Co-authored-by: Justin Nguyen <70992422+jnvtnguyen@users.noreply.github.com>

Update Acceptance-Tests.md

a67880a

Co-authored-by: Justin Nguyen <70992422+jnvtnguyen@users.noreply.github.com>

Update Acceptance-Tests.md

7cdb4f5

Co-authored-by: Justin Nguyen <70992422+jnvtnguyen@users.noreply.github.com>

add console-reporter.ts description

4d08f2b

link updated

316752c

Merge branch 'develop' of https://github.com/rahat2134/oppia-web-deve…

3a57ef1

…loper-docs into Acceptance-Test-doc-update

Merge branch 'develop' of https://github.com/rahat2134/oppia-web-deve…

2cca8a0

…loper-docs into Acceptance-Test-doc-update

Merge branch 'Acceptance-Test-doc-update' of https://github.com/rahat…

4f965e9

…2134/oppia-web-developer-docs into Acceptance-Test-doc-update

Merge branch 'develop' into Acceptance-Test-doc-update

9b3c931

rahat2134 requested a review from jnvtnguyen April 14, 2024 07:50

seanlip assigned jnvtnguyen Apr 18, 2024

jnvtnguyen requested changes Apr 18, 2024

View reviewed changes

jnvtnguyen removed their assignment Apr 18, 2024

rahat2134 and others added 3 commits April 22, 2024 21:47

changes

93fce11

Merge branch 'oppia:develop' into Acceptance-Test-doc-update

101ec2e

Merge branch 'Acceptance-Test-doc-update' of https://github.com/rahat…

4f8c56a

…2134/oppia-web-developer-docs into Acceptance-Test-doc-update

rahat2134 requested a review from jnvtnguyen April 22, 2024 16:22

jnvtnguyen requested changes Apr 22, 2024

View reviewed changes

change

fdac189

rahat2134 requested a review from jnvtnguyen April 22, 2024 17:47

jnvtnguyen approved these changes Apr 22, 2024

View reviewed changes

change

19e70f2

Ash-2k3 assigned seanlip Apr 24, 2024

seanlip requested changes Apr 24, 2024

View reviewed changes

seanlip removed their assignment Apr 24, 2024

change

2b2729c

rahat2134 requested a review from seanlip April 24, 2024 12:58

seanlip requested changes Apr 24, 2024

View reviewed changes

change

7cf2e96

rahat2134 requested a review from seanlip April 24, 2024 13:22

seanlip approved these changes Apr 24, 2024

View reviewed changes

seanlip merged commit f9502cf into oppia:develop Apr 24, 2024
3 checks passed

rahat2134 deleted the Acceptance-Test-doc-update branch April 24, 2024 13:37


		In this example, the `discardCurrentChanges()` function checks if the viewport width corresponds to a mobile device, and if so, clicks on the mobile-specific discard button. Otherwise, it clicks on the desktop-specific discard button. Finally, it confirms the discard action. This approach allows us to maintain a single set of tests while accommodating differences between desktop and mobile environments.

		### console errors logging functionality in Acceptance Tests


		### console errors logging functionality in Acceptance Tests

		Now, Acceptance Tests are capable of finding console errors during CUJ's and it will break the test. But in some cases where the console error can be ignored and not in high priority right now then we can ignore that with help of `consoleReporter.setConsoleErrorsToIgnore`


		We can handle these differences by including conditional statements in the `utils` file, using the `isViewportAtMobileWidth()` function to execute commands specific to mobile devices.

		For Eg.

	To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which have the details of the issue. If the error is new, also raise it in the issue section.
	To handle errors that need to be fixed, you should include them directly within the `console-reporter.ts` utility. In this file, you would add the error regex to the `CONSOLE_ERRORS_TO_FIX ` array and add a TODO comment which points to the existing issue number. If the error doesn't have any corresponding issue, then file a new issue on our [issue tracker](https://github.com/oppia/oppia/issues).


		Similar to desktop, we also have acceptance tests for mobile to ensure responsiveness and uninterrupted user journeys on small screen devices. While the tests themselves remain largely the same for both desktop and mobile, there are some differences. For instance, large full menus on desktop may be converted to dropdowns, hamburger menus, or other shortcuts on mobile, requiring additional actions to complete the tests.

		### How to Write Tests for Mobile

Acceptance test doc update #297

Acceptance test doc update #297

Conversation

rahat2134 commented Mar 28, 2024

rahat2134 commented Mar 28, 2024

jnvtnguyen left a comment

Choose a reason for hiding this comment

jnvtnguyen Mar 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnvtnguyen Apr 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahat2134 commented Apr 15, 2024

rahat2134 commented Apr 18, 2024

jnvtnguyen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahat2134 commented Apr 22, 2024

Choose a reason for hiding this comment

rahat2134 Apr 22, 2024 • edited Loading

Choose a reason for hiding this comment

jnvtnguyen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahat2134 Apr 22, 2024 • edited Loading

Choose a reason for hiding this comment

rahat2134 commented Apr 22, 2024

Ash-2k3 commented Apr 24, 2024

seanlip left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahat2134 commented Apr 24, 2024

seanlip left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seanlip left a comment

Choose a reason for hiding this comment

jnvtnguyen Mar 30, 2024 •

edited

Loading

jnvtnguyen Apr 3, 2024 •

edited

Loading

rahat2134 Apr 22, 2024 •

edited

Loading

rahat2134 Apr 22, 2024 •

edited

Loading