Regression Tests - Keyman for Mac #6436

MakaraSok · 2022-03-28T03:53:36Z

Keyman for Mac Acceptance Test Procedures

These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
Copy these lists of tests into a new issue (for tier transitions) or a comment on the PR.
Use User Testing format for documenting tests.

User Testing

Setup Steps

Ensure the device is running the latest version of macOS (beta build for beta phase, alpha build for alpha phase
Uninstall previous version of Keyman on the device
Install latest master build (or whatever version is to be tested) of Keyman on the device

SUITE_BASIC_SMOKE_TEST: Basic smoke test

TEST_VERSION: Verify that the version displayed in the About box is correct.
TEST_DOWNLOAD_WIFI_OFF:
- On device, turn Wifi off and/or disconnect Ethernet cable
- Activate Keyman and select Keyman Configuration from the input methods menu. Click Download keyboard
- Verify error message "Could not reach Keyman server!" is displayed (This is currently not happening. Bug reported as: bug(mac): Searching for a Keyboard to download in Keyman for macOS gives no feedback when Internet is unavailable #302)
- Close Keyman configuration windows.
TEST_DOWNLOAD_WIFI_ON:
- On device, re-enable Internet connectivity
- Activate Keyman and select Keyman Configuration from the input methods menu. Click Download keyboard.
- Verify that page displays correctly (should have a Keyboard Search box with the prompt: Enter language or keyboard.
- Search for Amharic. Should see several keyboards, including some by The Ge'ez Frontier Foundation and some by SIL.
- Double-click any KMP file (in Finder). Should see a “Do you want the Keyman to install the keyboard from the file 'xxx.kmp'?“ message. If you click Install, the package should be installed (or updated) and the configuration window should display, showing that the keyboard(s) in the package are installed and enabled. If you click Cancel, the configuration window should not be displayed (but there is a known issue that the very first time after Keyman loads, if the configuration window has not been shown, it will be displayed even if the user clicks Cancel).

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_URL_BAR: Chrome browser URL bar
GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control
GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)
GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online
Note: Not using the Office Extension for Chrome - not sure if that would make a difference
- Beginning of paragraph (currently known compatibility issues steps 1, 3, 4 & 5)
- Elsewhere in paragraph
GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)
GROUP_SAFARI_URL_BAR: Safari browser URL bar
GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control
GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)
GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online
a) Beginning of paragraph (currently known compatibility issues steps 1, 3, 4 & 5, 6&7)
b) Elsewhere in paragraph (occasionally steps 4 & 5 deletes a preceding (space?) character, but I haven't figured out when/why. Step 6 is not relevant. Step 7 fails for the same reason it doesn't work at the beginning of the paragraph. There seems to be a certain level of general twitchiness, with extra characters sometimes being displayed - sometimes only briefly - and characters being deleted.)
GROUP_FIREFOX_URL_BAR: Firefox browser URL bar
GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control
GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)
GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online
a) Beginning of paragraph
b) Elsewhere in paragraph
GROUP_MESSAGES_APP: Messages app
a) New message with nobody filled in in To: line
b) New message in existing conversation, such that the message area has grayed out text: "iMessage")
GROUP_NOTES_APP: Notes app
GROUP_TEXTEDIT: TextEdit
GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 6 & 7. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)
GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).
GROUP_MAIL: Mail (message body)

Test Cases

Select the Amharic keyboard. For each of the following applications/contexts, test that

TEST_AMHARIC:
(1). typing ta produces ታ;
(2). typing t, left-arrow, a produces አት;
(3). typing tt, left-arrow, a produces ታት;
(4). typing tt, then mouse-clicking between the two ታ characters, and then typing a produces ታት.

In each of the last three cases, the insertion point should end up before the final ት character.
TEST_SHORTCUTS፡ (cont. from above)
(5). common keyboard shortcuts for the app work as expected. For example:
- ⌘-F (typically opens a Find dialog or control);
- ⌘-O (File Open);
- ⌘-X (Cut);
- ⌘-C (Copy);
- ⌘-V (Paste);
- ⌘-A (Select All).
(Note that in Google Docs, these commands correspond to both a Google-Docs command and a browser command. The "normal" behavior in all three browsers is to activate the Google Docs command, not the browser command, with the exception that ⌘-O in Safari opens Safari's standard macOS Open file dialog.)

(6). with a few characters displayed, ⌘-A followed by typing t replaces selected characters with ት.
(7). typing at, then ⌘-A, followed by a produces አ.

SUITE_OSK: OSK UI/Functionality Tests

TEST_OSK_UI:
- Verify behavior of "Always show on-screen keyboard" option
- Verify behavior and appearance of On-Screen Keyboard and the menu that toggles it on and off.
- With OSK visible, verify that switching keyboards updates the display to show the appropriate layout.
- Verify that switching to system keyboard or other IM turns off Keyman OSK.

keymanapp-test-bot · 2022-03-28T03:53:40Z

User Test Results

Test specification and instructions

🟩 SUITE_BASIC_SMOKE_TEST: Basic smoke test

✅ TEST_VERSION (PASSED): 15.0.244-beta
🟩 TEST_DOWNLOAD_WIFI_OFF (SKIPPED): This test is to be included in the future test if this issue (bug(mac): Searching for a Keyboard to download in Keyman for macOS gives no feedback when Internet is unavailable #302) is fixed.
✅ TEST_DOWNLOAD_WIFI_ON (PASSED): No issue found.

🟩 SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

✅ GROUP_CHROME_URL_BAR: Chrome browser URL bar
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): All work as expected.
- ✅ TEST_SHORTCUTS (PASSED): All work as expected.
🟩 GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control
- ✅ TEST_AMHARIC (PASSED): All work as expected.
- 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in FB search control on Chrome #6655) is fixed.
✅ GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- ✅ TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.
🟩 GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in Word Online on Chrome and Google Docs on Safari #6656) is fixed.
✅ GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- ✅ TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.
✅ GROUP_SAFARI_URL_BAR: Safari browser URL bar
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
✅ GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
🟩 GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in Word Online on Chrome and Google Docs on Safari #6656) is fixed.
🟩 GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online
- ✅ TEST_AMHARIC (PASSED): No issue found.
- 🟩 TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in Word Online on Safari #6657) is fixed.
✅ GROUP_FIREFOX_URL_BAR: Firefox browser URL bar
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
✅ GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
✅ GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- ✅ TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.
✅ GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
✅ GROUP_MESSAGES_APP: Messages app
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
- ✅ TEST_SHORTCUTS (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
✅ GROUP_NOTES_APP: Notes app
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
- ✅ TEST_SHORTCUTS (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
✅ GROUP_TEXTEDIT: TextEdit
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
✅ GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 6 & 7. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- ✅ TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.
✅ GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
- ✅ TEST_SHORTCUTS (PASSED): No issue found.
✅ GROUP_MAIL: Mail (message body)
2 tests PASSED
- ✅ TEST_AMHARIC (PASSED): No issue found.
- ✅ TEST_SHORTCUTS (PASSED): No issue found. (Command+O is N/A.)

🟩 SUITE_OSK: OSK UI/Functionality Tests

🟩 TEST_OSK_UI (SKIPPED): This test is to be included in the future test if this issue (bug(mac): Shift, AltGr and Shift AltGr don't output the expected characters when triggered from the OSK #6578) is fixed.

bharanidharanj · 2022-04-01T13:23:43Z

SUITE_BASIC_SMOKE_TEST: Basic smoke test

TEST_VERSION (PASSED): Successfully installed Keyman 16.0.4 build from Master branch. (in Mac OS 12.1)

TEST_DOWNLOAD_WIFI_OFF (PASSED): Did not see any error message but it is showing blank window.

TEST_DOWNLOAD_WIFI_ON (PASSED): Successfully downloaded and installed Amharic Keyboard.

bharanidharanj · 2022-04-01T14:08:16Z

GROUP_NOTES_APP: Notes app

TEST_AMHARIC (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Notes App and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Notes App and it is working as expected.

GROUP_TEXTEDIT: TextEdit

TEST_AMHARIC (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Text Editor and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this with the Keyman 16.0.4 Alpha build in Text Editor and it is working as expected.

bharanidharanj · 2022-04-01T15:03:36Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_URL_BAR: Chrome browser URL bar

TEST_AMHARIC (PASSED): Tested this on Chrome browser in Mac 12.1 Monterey OS and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this on Chrome browser in Mac 12.1 Monterey OS and it is working as expected.

bharanidharanj · 2022-04-04T10:43:06Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_URL_BAR: Safari browser URL bar

TEST_AMHARIC (PASSED): Tested this in Safari (Mac Mini Version 12.1 Monterey) with the latest Keyman 16.0.4 build and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this in Safari (Mac Mini Version 12.1 Monterey) with the latest Keyman 16.0.4 build and it is working as expected.

bharanidharanj · 2022-04-04T11:43:27Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

TEST_AMHARIC (PASSED): Tested this in FireFox Browser URL bar and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this in FireFox Browser URL bar and it is working as expected.

bharanidharanj · 2022-04-04T12:21:44Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

TEST_AMHARIC (FAILED): Tested this on Atom - Snippet editor and it the compatibility issue remains for steps 3 and 4. Step 5 seems to be working fine with the current build. (16.0.4)

Compatibility issue in step 3 and 4:

TEST_SHORTCUTS (PASSED): Tested this on Atom - Snippet editor and it seems to be working as expected. I did not see any compatibility issue

bharanidharanj · 2022-04-04T12:58:23Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_NOTES_APP: Notes app

TEST_AMHARIC (PASSED): Tested this on Mac Mini OS, NOTES App and it seems to be working as expected.
TEST_SHORTCUTS (PASSED): Tested this on Mac Mini OS, NOTES App and all the commands are working as expected. (except command + O)I think this one is working only on Safari browser.

bharanidharanj · 2022-04-04T13:05:37Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_TEXTEDIT: TextEdit

TEST_AMHARIC (PASSED): Tested this with the latest Keyman 16.0.4 build and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this with the latest Keyman 16.0.4 build and it is working as expected.

bharanidharanj · 2022-04-04T14:40:14Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_MESSAGES_APP: Messages app

TEST_AMHARIC (PASSED): Tested this on the Messages App with the latest Keyman 16.0.4 build and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this on the Messages App with the latest Keyman 16.0.4 build and it is working as expected.

bharanidharanj · 2022-04-06T10:20:52Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

TEST_AMHARIC (PASSED): Tested in Chrome browser Google Document and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested in Chrome browser Google Document and it is working as expected.

bharanidharanj · 2022-04-06T10:39:57Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

TEST_AMHARIC (PASSED): Tested this on Facebook Search bar and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this on Facebook Search bar and it is working as expected.

bharanidharanj · 2022-04-06T11:14:30Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

TEST_AMHARIC (PASSED): Tested this on Facebook Search Control bar and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this on Facebook Search Control bar and it is working as expected.

GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

TEST_AMHARIC (PASSED): Tested this on Facebook Search Control bar and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this on Facebook Search Control bar and it is working as expected.

bharanidharanj · 2022-04-06T11:34:47Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

TEST_AMHARIC (PASSED): Tested this in LibreOffice 7.3.1 and it is working fine, except the compatibility issues in Steps 3,4 and 5.
TEST_SHORTCUTS (PASSED): Tested this LibreOffice 7.3.1 and the Shortcut keys are working as expected.

bharanidharanj · 2022-04-06T11:47:56Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_MAIL: Mail (message body)

TEST_AMHARIC (PASSED): Tested this on Mail (Message body) and it is working as expected.
TEST_SHORTCUTS (PASSED): Tested this on Mail (Message body) and it is working as expected.

bharanidharanj · 2022-04-06T12:01:15Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 7 & 8. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)

TEST_AMHARIC (PASSED): Tested this as per the acceptance test and it working as expected in the terminal.
TEST_SHORTCUTS (PASSED): Tested this as per the acceptance test and it working as expected in the terminal.

bharanidharanj · 2022-04-06T12:13:26Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

SUITE_OSK: OSK UI/Functionality Tests

TEST_OSK_UI (PASSED): Tested Keyman On Screen Keyboard functionality, as per the test steps mentioned in the test suite and it is working as expected.

bharanidharanj · 2022-04-06T12:27:03Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online

TEST_AMHARIC (FAILED): Tested this in Word Online (Office.com)in Chrome browser and it failed in 1, 3, 4, 6 and 7 Steps.
TEST_SHORTCUTS (PASSED): Tested this in Word Online in Chrome browser and the Shortcut keys are working as expected.

bharanidharanj · 2022-04-06T12:41:58Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online

TEST_AMHARIC (PASSED): Tested this in Firefox_Word Online (from Office.com) and it seems to be working fine.
TEST_SHORTCUTS (PASSED): Tested this in Firefox_Word Online (from Office.com) and it seems to be working fine.

bharanidharanj · 2022-04-06T12:54:31Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online

TEST_AMHARIC (PASSED): Tested this in Word Online document and it working fine except the known compatibility issues as it mentioned in the User Testing.
TEST_SHORTCUTS (PASSED): Tested this in Word Online document and it is working as expected.

bharanidharanj · 2022-04-06T13:04:13Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

TEST_AMHARIC (PASSED): Tested this on Google document and it is working fine except the know compatibility issue for steps 3 and 4.
TEST_SHORTCUTS (PASSED): Tested this on Google document and it is working as expected.

bharanidharanj · 2022-04-06T13:09:29Z

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

TEST_AMHARIC (PASSED): Tested this in Google online document and it is working fine except the known compatibility issue for steps 3,4 & 5.
TEST_SHORTCUTS (PASSED): Tested this in Good Online document and it is working as expected.

MakaraSok · 2022-05-10T07:03:57Z

To be tested with the latest beta build

@keymanapp-test-bot retest all

MakaraSok · 2022-05-10T09:52:08Z

Test Results

Tested with Keyman 15.0.244-beta on macOS 12.3.1.

SUITE_BASIC_SMOKE_TEST: Basic smoke test

TEST_VERSION (PASSED): 15.0.244-beta
TEST_DOWNLOAD_WIFI_OFF (FAILED): a blank page instead of the error message is shown
TEST_DOWNLOAD_WIFI_ON (PASSED): No issue found.

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_URL_BAR: Chrome browser URL bar

TEST_AMHARIC (PASSED): All work as expected.
TEST_SHORTCUTS (PASSED): All work as expected.

GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

TEST_AMHARIC (PASSED): All work as expected.
TEST_SHORTCUTS (FAILED): (7). outputs ታ instead of አ.

GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (FAILED): (6). and (7). output t and a instead of ት and አ.

GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_SAFARI_URL_BAR: Safari browser URL bar

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (FAILED): (6). and (7). output t and a instead of ት and አ.

GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (FAILED): (6). and (7). remove the selected text, but output nothing.

MakaraSok · 2022-05-11T02:41:10Z

Test Results

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_MESSAGES_APP: Messages app

TEST_AMHARIC (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
TEST_SHORTCUTS (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.

GROUP_NOTES_APP: Notes app

TEST_AMHARIC (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.
TEST_SHORTCUTS (PASSED): No issue found except, ⌘-O doesn't open file/Finder. Which doesn't stop one from using tha app.

GROUP_TEXTEDIT: TextEdit

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_TERMINAL: Terminal (known compatibility issues: Steps 1 & 3 insert an extra leading space if there are no preceding characters. Skip step 4 - mouse doesn't move insertion point in Terminal. Step 5 can be done using left-arrow instead. Skip steps 6 & 7. If the Terminal window has LOTS of text in it, it won't provide any context, which leads to other compatibility problems.)

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (PASSED): No issue found except the ones mentioned above.

GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

TEST_AMHARIC (PASSED): No issue found except the ones mentioned above.
TEST_SHORTCUTS (PASSED): No issue found.

GROUP_MAIL: Mail (message body)

TEST_AMHARIC (PASSED): No issue found.
TEST_SHORTCUTS (PASSED): No issue found. (Command+O is N/A.)

SUITE_OSK: OSK UI/Functionality Tests

TEST_OSK_UI (BLOCKED): Please clarify if the OSK should behave like that of the physical keyboard when typing/pressing on the key caps. For time being, they are different. Please see the table below.

keystrokes	physical keyboard	OSK
`ta`	ታ	ትአ

MakaraSok · 2022-05-26T03:44:51Z

SUITE_BASIC_SMOKE_TEST

TEST_DOWNLOAD_WIFI_OFF (SKIPPED): This test is to be included in the future test if this issue (bug(mac): Searching for a Keyboard to download in Keyman for macOS gives no feedback when Internet is unavailable #302) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_CHROME_FB_SEARCH_CONTROL

TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in FB search control on Chrome #6655) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_CHROME_WORD_ONLINE

TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in Word Online on Chrome and Google Docs on Safari #6656) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_SAFARI_GOOGLE_DOCS

TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in Word Online on Chrome and Google Docs on Safari #6656) is fixed.

SUITE_APPLICATION_COMPATIBILITY

GROUP_SAFARI_WORD_ONLINE

TEST_SHORTCUTS (SKIPPED): This test is to be included in the future test if this issue (bug(mac): wrong output in Word Online on Safari #6657) is fixed.

MakaraSok · 2022-05-26T08:53:49Z

SUITE_OSK

TEST_OSK_UI (SKIPPED): This test is to be included in the future test if this issue (bug(mac): Shift, AltGr and Shift AltGr don't output the expected characters when triggered from the OSK #6578) is fixed.

keymanapp-test-bot bot added has-user-test user-test-required User tests have not been completed labels Mar 28, 2022

MakaraSok changed the title ~~Test Formatting for Keyman for Mac ATP Only [please don't test this]~~ Regression Tests - Keyman for Mac Mar 30, 2022

mcdurdin added the mac/ label Mar 30, 2022

mcdurdin added this to the B15S5 milestone Mar 30, 2022

mcdurdin modified the milestones: B15S5, B15S6 Apr 2, 2022

keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Apr 6, 2022

mcdurdin modified the milestones: B15S6, B15S7 Apr 15, 2022

mcdurdin modified the milestones: B15S7, A16S1 May 3, 2022

keymanapp-test-bot bot added the user-test-required User tests have not been completed label May 10, 2022

keymanapp-test-bot bot removed the user-test-required User tests have not been completed label May 11, 2022

mcdurdin modified the milestones: A16S1, A16S2 May 15, 2022

This was referenced May 25, 2022

bug(mac): wrong output in FB search control on Chrome #6655

Open

bug(mac): wrong output in Word Online on Chrome and Google Docs on Safari #6656

Open

bug(mac): wrong output in Word Online on Safari #6657

Open

MakaraSok closed this as completed May 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Regression Tests - Keyman for Mac #6436

Regression Tests - Keyman for Mac #6436

MakaraSok commented Mar 28, 2022 •

edited

keymanapp-test-bot bot commented Mar 28, 2022 •

edited

bharanidharanj commented Apr 1, 2022

bharanidharanj commented Apr 1, 2022

bharanidharanj commented Apr 1, 2022

bharanidharanj commented Apr 4, 2022

bharanidharanj commented Apr 4, 2022

bharanidharanj commented Apr 4, 2022

bharanidharanj commented Apr 4, 2022

bharanidharanj commented Apr 4, 2022

bharanidharanj commented Apr 4, 2022

bharanidharanj commented Apr 6, 2022 •

edited

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022 •

edited

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

bharanidharanj commented Apr 6, 2022

MakaraSok commented May 10, 2022

MakaraSok commented May 10, 2022 •

edited

MakaraSok commented May 11, 2022

MakaraSok commented May 26, 2022

MakaraSok commented May 26, 2022 •

edited

Regression Tests - Keyman for Mac #6436

Regression Tests - Keyman for Mac #6436

Comments

MakaraSok commented Mar 28, 2022 • edited

Keyman for Mac Acceptance Test Procedures

User Testing

Setup Steps

SUITE_BASIC_SMOKE_TEST: Basic smoke test

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

Test Cases

SUITE_OSK: OSK UI/Functionality Tests

keymanapp-test-bot bot commented Mar 28, 2022 • edited

User Test Results

🟩 SUITE_BASIC_SMOKE_TEST: Basic smoke test

🟩 SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

🟩 SUITE_OSK: OSK UI/Functionality Tests

bharanidharanj commented Apr 1, 2022

SUITE_BASIC_SMOKE_TEST: Basic smoke test

bharanidharanj commented Apr 1, 2022

GROUP_NOTES_APP: Notes app

GROUP_TEXTEDIT: TextEdit

bharanidharanj commented Apr 1, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_URL_BAR: Chrome browser URL bar

bharanidharanj commented Apr 4, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_URL_BAR: Safari browser URL bar

bharanidharanj commented Apr 4, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_URL_BAR: Firefox browser URL bar

bharanidharanj commented Apr 4, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_ATOM: Atom editor - test in Snippet editor (currently known compatibility issue for steps 3, 4 & 5)

bharanidharanj commented Apr 4, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_NOTES_APP: Notes app

bharanidharanj commented Apr 4, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_TEXTEDIT: TextEdit

bharanidharanj commented Apr 4, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_MESSAGES_APP: Messages app

bharanidharanj commented Apr 6, 2022 • edited

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_GOOGLE_DOCS: Chrome browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_FB_SEARCH_CONTROL: Chrome browser Facebook search control

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_FB_SEARCH_CONTROL: Safari browser Facebook search control

GROUP_FIREFOX_FB_SEARCH_CONTROL: Firefox browser Facebook search control

bharanidharanj commented Apr 6, 2022 • edited

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_LIBREOFFICE: LibreOffice 7.0 (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - LibreOffice Vanilla can't report context.).

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_MAIL: Mail (message body)

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

SUITE_OSK: OSK UI/Functionality Tests

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_CHROME_WORD_ONLINE: Chrome browser Word Online

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_WORD_ONLINE: Firefox browser Word Online

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_WORD_ONLINE: Safari browser Word Online

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_FIREFOX_GOOGLE_DOCS: Firefox browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

bharanidharanj commented Apr 6, 2022

SUITE_APPLICATION_COMPATIBILITY: Application Compatibility Tests

GROUP_SAFARI_GOOGLE_DOCS: Safari browser Google Docs document (currently known compatibility issue for steps 3, 4 & 5 because any selection change will cause the context to be reset - Google docs can't report context.)

MakaraSok commented May 10, 2022

MakaraSok commented May 10, 2022 • edited

MakaraSok commented Mar 28, 2022 •

edited

keymanapp-test-bot bot commented Mar 28, 2022 •

edited

bharanidharanj commented Apr 6, 2022 •

edited

bharanidharanj commented Apr 6, 2022 •

edited

MakaraSok commented May 10, 2022 •

edited

MakaraSok commented May 26, 2022 •

edited