Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(Android): Regression Test for Alpha 16.0.49 #7099

Closed
3 tasks
MakaraSok opened this issue Aug 23, 2022 · 25 comments
Closed
3 tasks

test(Android): Regression Test for Alpha 16.0.49 #7099

MakaraSok opened this issue Aug 23, 2022 · 25 comments
Labels
android/ has-user-test test Any acceptance test issue user-test-failed
Milestone
A16S13

Comments

@MakaraSok
Copy link
Collaborator

MakaraSok commented Aug 23, 2022 •
edited by mcdurdin

Keyman for Android Regression Test Procedures

  • These test procedures are to be run each sprint, also before moving from alpha to beta, or beta to stable, and ideally before PRs are merged into stable branches.

  • Copy these checklists of tests into a new issue and title/tag as required.

Gather Assets for Testing

  • Physical Android phone (record device's Android -- versions 5, 8 and 11)
  • External Bluetooth or USB QWERTY keyboard
  • External Bluetooth or USB AZERTY keyboard (European AZERTY layout with 102nd key)

Setup Steps

  1. Uninstall previous version of Keyman on the device (don't forget to reset user settings for the app; in Android Pie: Settings>Storage>Other apps>Keyman>Clear Data)
  2. Install test build APK off of the current latest alpha 16.0.49
  3. Accept all the Android permission requests for storage (applies to Android 6.0 Marshmallow onward)

User Testing

SUITE_INSPECT: Visual Inspection of master branch

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

click to expand

Various versions of Androids

UI for these tests assume default English locale

  • GROUP_ANDROID_5
  • GROUP_ANDROID_8
  • GROUP_ANDROID_11

  • TEST_GET_STARTED: This tests the "Get Started" menu from a fresh installation

    Expand to see how to do and verify this
    1. When the app loads, verify the "Get Started" menu is visible
    2. From the "Get Started" menu, touch "Add a keyboard for your language"
    3. Verify "Install Keyboard or Dictionary" menu appears
    4. From the "Install Keyboard or Dictionary" menu, click the back arrow to return to "Get Started"
    5. From the "Get Started" menu, touch "Enable Keyman as system-wide keyboard"
    6. Verify Android system menus appear for controlling virtual keyboard
    7. Scroll down and enable Keyman as a system keyboard.
    8. On the "Attention" dialog, read and hit "OK" to dismiss the dialogs.
    9. Touch the back button to return to "Get Started"
    10. Verify "Enable Keyman as system-wide keyboard" now has a ticked checkbox.
    11. From the "Get Started" menu, touch "Set Keyman as default keyboard"
    12. On the "Change keyboard" dialog, select Keyman
    13. Verify on the "Get Started" menu that "Set Keyman as default keyboard" now has a ticked checkbox.
    14. From the "Get Started" menu, touch "More info"
    15. Verify the Info page appears with a version string at the bottom of the page
    16. Touch the back button (arrow) to return to "Get Started"
    17. From the "Get Started" menu, untick the last option 'Show "Get Started" on startup'
    18. Close the "Get Started" menu and exit the app.
    19. From the Android device, start Keyman for Android
    20. Verify the "Get Started" menu does not appear.

  • TEST_INSTALL_KD: This menu is accessed via Get Started menu or the Settings button-->"Install Keyboard or Dictionary"

    1. On the device, enable internet connectivity
    2. Touch "Install from keyman.com"
    3. Verify a new keyboard can be downloaded and selected
    4. On the device, disable internet connectivity
    5. From "Install Keyboard or Dictionary", touch "Install from keyman.com"
    6. Verify device displays a message "Cannot connect to Keyman server!"
    7. On the device, re-enable internet connectivity

  • TEST_KEYMAN_SETTTINGS:

    Expand to see how to do and verify this
    1. From "Settings" --> "Installed Languages"
    2. Select "English" --> English Settings
    3. Select keyboard settings menu
    4. Verify Info appears with keyboard version, help link, and QR code
    5. If keyboard update is available, click on version and verify update
    6. If help link is available, click on link
    7. Verify keyboard help documentation appears
    8. Return to English Settings menu
    9. Select Dictionary --> English dictionary (MTNT)
    10. Verify the Dictionary info appears
    11. Return to English Settings menu
    12. Turn on/off 'Enable Predictions' and validate each
    13. Turn on/off 'Enable Corrections' and validate each. For example, when Corrections are enabled and context "tooo" should still suggest "too"
    14. Validate that when both predictions and corrections are off, banner is not visible
    15. Add/Remove dictionaries - validate
    16. If multiple dictionaries are available, test swapping between them

  • TEST_SHARE: Type and verify text can be shared to external app

  • TEST_KEYMAN_BROWSER: Keyman Browser ###

    1. On default page, click on text field and set system keyboard to Keyman
    2. Verify can type with Keyman as system keyboard
    3. Close and reopen Keyman app

  • TEST_TEXT_SIZE: Verify text can be rescaled from Text Size 16 to 72

  • TEST_CLEAR_TEXT: Verify text can be cleared

  • TEST_INFO:

    1. From the device, disable internet access
    2. From the Keyman app menu, select "Info"
    3. Verify Keyman for Android version appears at the bottom of the screen
    4. Verify Keyman for Android help documentation is displayed.
    5. From the device, enable internet access

  • TEST_INSTALL_UPDATES: This menu option only appears when a language resource (keyboard or lexical-model) update is available

    Expand to see how to do and verify this
    1. Download an old version of sil_cameroon_qwerty
    2. On the device, set airplane mode on (turns off internet access)
    3. On the device, go to settings, and manually set the date to 1 year in the past (e.g. May 2021)
    4. Start Keyman for Android
    5. From the Keyman Settings, install the local kmp file sil_cameroon_qwerty.kmp (will be an old version 6.0.4)
    6. Close Keyman
    7. On the device, go to settings and set the date to automatically update from internet
    8. On the device, turn off airplane mode (turns on internet access)
    9. On the device, go to settings and verify the date is updated
    10. Launch Keyman for Android
    11. Dismiss "Get Started" and wait for the catalog to update
    12. After a while, there should be a notification of a keyboard update being available
    13. At the bottom of the Keyman overflow menu, verify keyboard update is available
    14. Click on "Install Updates"
    15. Accept "Download" on the dialogue
    16. Verify resource is updated after background download complete

  • TEST_CHANGE_DISPLAY_LANG: This tests changing the display language for the App and assumes starting with English locale

    1. From "Settings" --> "Display Language"
    2. Select "Khmer"
    3. From "Settings" --> Verify menu selections in Khmer
    4. From the overflow menu --> clear text
    5. Verify Keyman text field has Khmer prompt (for "Start typing here")
    6. From "Settings" --> "Change Display Language"
    7. If the string is translated, the first row should be "Default Locale" in Khmer
    8. Select "English"
    9. Verify the app reloads in English

  • TEST_ADJUST_K_HEIGHT: This menu option allows the user to adjust the OSK height for the current orientation (portrait or landscape). The customized height is saved as a preference

    Expand to see how to do and verify this
    1. Have the device in portrait orientation
    2. From "Settings" --> "Adjust keyboard height"
    3. On the menu, drag the OSK height to change the keyboard height
    4. Click back
    5. Verify the OSK is refreshed to the selected height
    6. From "Settings" --> "Adjust keyboard height"
    7. Click the "reset to defaults" button and then click back
    8. Verify the OSK reverts to the original height
    9. Rotate the device to landscape orientation
    10. From "Settings" --> "Adjust keyboard height"
    11. On the menu, drag the OSK height to change the keyboard height
    12. Click back
    13. Verify the OSK is refreshed to the selected height
    14. From "Settings" --> "Adjust keyboard height"
    15. Click the "reset to defaults" button and then click back
    16. Verify the OSK reverts to the original height
    17. Rotate the device back to portrait orientation

  • TEST_SPACEBAR_CAPTION: This menu option allows the user to adjust the label displayed on the keyboard spacebar

    Expand to see how to do and verify this
    1. Select the default SIL EuroLatin keyboard
    2. From "Settings" --> Spacebar caption
    3. On the menu, select "Language"
    4. Click back until the OSK is displayed
    5. Verify the spacebar label is "English"
    6. From "Settings" --> Spacebar caption
    7. On the menu, select "Keyboard"
    8. Click back until the OSK is displayed
    9. Verify the spacebar label is "EuroLatin (SIL)"
    10. From "Settings" --> Spacebar caption
    11. On the menu, select "Blank"
    12. Click back until the OSK is displayed
    13. Verify the spacebar label is blank
    14. From "Settings" --> Spacebar caption
    15. On the menu, select "Language+Keyboard"
    16. Click back until the OSK is displayed
    17. Verify the spacebar label is "English - EuroLatin (SIL)"

SUITE_KEYBOARD_FUNCTIONALITY: Keyboard Tests

click to expand

Various versions of Androids

UI for these tests assume default English locale

  • GROUP_ANDROID_5
  • GROUP_ANDROID_8
  • GROUP_ANDROID_11

In-app Keyboards

These tests are run in the context of typing in the Keyman app

click to expand

  • TEST_INAPP_LATIN_KEYBOARD_PORTRAIT: English(EuroLatin SIL)
    In portrait orientation, verify OSK is visible and fills the width the bottom of the screen

    1. Verify long-press q key works
    2. Verify long-press p key works
    3. Verify uppercase layer can be selected via SHIFT
    4. Verify number layer can be selected via 123
    5. Verify long-press 1 key works
    6. Verify long-press 0 key works
    7. Verify backspace, space, and enter keys work
    8. Verify Khmer -> Khmer Angkor keyboard can be added via Settings menu

  • TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE: English(EuroLatin SIL)
    In portrait orientation, verify OSK is visible and fills the width the bottom of the screen

    1. Verify long-press q key works
    2. Verify long-press p key works
    3. Verify uppercase layer can be selected via SHIFT
    4. Verify number layer can be selected via 123
    5. Verify long-press 1 key works
    6. Verify long-press 0 key works
    7. Verify backspace, space, and enter keys work
    8. Verify Khmer -> Khmer Angkor keyboard can be added via Settings menu

  • TEST_INAPP_LONG_PRESS: English(EuroLatin SIL)

    Type on the OSK using the following scenarios and verify expected output:

    1. Clicking a suggestion on the suggestion banner - should insert the suggestion
    2. short-press a key and release - should insert the base key
    3. long-press a key, select a long-press key, and release - should insert the long-press key
    4. long-press a key, while keeping the finger down, move off the long-press options, and release - should not output
    5. long-press a key, while keeping the finger down, move off the long-press options, then move back on a long-press option so it's highlighted, and release - should output the long-press key
    6. quickly type a long paragraph (e.g. repeat the word "reply") - verify long-press keys don't get stuck (displayed when not touching a key)

  • TEST_INAPP_NON_LATIN_KEYBOARD: Non-Latin script Keyboard

    1. In the app, add/select a random non-Latin script keyboard
    2. Verify OSK is visible
    3. Test several keys and modifiers and verify Latin characters are not appearing.

  • TEST_INAPP_ROTATE_P-TO-L:

    1. With Keyman not active, load it in a portrait orientation.
    2. Once loaded, rotate the device to a landscape orientation.
      • On Android devices, you may need to press something like this for the rotation to occur:
      • The test "passes" if the OSK rotates properly.

  • TEST_INAPP_ROTATE_L-TO-P:

    1. With Keyman not active, load it in a landscape orientation.
    2. Once loaded, rotate the device to a portrait orientation.
      • On Android devices, you may need to press something like this for the rotation to occur:
      • The test "passes" if the OSK rotates properly.

System Keyboards

These tests are run with Keyman selected as a system keyboard, and in the context of typing in an external app (like Google Keep)

click to expand

  • TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT: English (EuroLatin SIL)
    In portrait orientation, verify OSK is visible and fills the width the bottom of the screen

    1. Verify long-press q key works
    2. Verify long-press p key works
    3. Verify uppercase layer can be selected via SHIFT
    4. Verify number layer can be selected via 123
    5. Verify long-press 1 key works
    6. Verify long-press 0 key works
    7. Verify backspace, space, and enter keys work
    8. Verify can change to random non-Latin script keyboard via globe button

  • TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE: English(EuroLatin SIL)
    In portrait orientation, verify OSK is visible and fills the width the bottom of the screen

    1. Verify long-press q key works
    2. Verify long-press p key works
    3. Verify uppercase layer can be selected via SHIFT
    4. Verify number layer can be selected via 123
    5. Verify long-press 1 key works
    6. Verify long-press 0 key works
    7. Verify backspace, space, and enter keys work
    8. Verify Khmer -> Khmer Angkor keyboard can be added via Settings menu

  • TEST_SYSTEM_LONG_PRESS: English(EuroLatin SIL)

    Type on the OSK using the following scenarios and verify expected output:

    1. Clicking a suggestion on the suggestion banner - should insert the suggestion
    2. short-press a key and release - should insert the base key
    3. long-press a key, select a long-press key, and release - should insert the long-press key
    4. long-press a key, while keeping the finger down, move off the long-press options, and release - should not output
    5. long-press a key, while keeping the finger down, move off the long-press options, then move back on a long-press option so it's highlighted, and release - should output the long-press key
    6. quickly type a long paragraph (e.g. repeat the word "reply") - verify long-press keys don't get stuck (displayed when not touching a key)

  • TEST_KEYBOARD_PICKER: Keyboard Picker menu to switch input method
    This tests the Keyboard Picker menu which lists other system input methods at the bottom of the menu. These appear only in Keyman system keyboard

    1. With only 1 Keyman keyboard installed and selected as a system keyboard, long-press and release the the globe button
    2. Verify the bottom of the Keyboards picker menu lists other available input methods
    3. Click on one of the other input methods
    4. Verify the Keyman system keyboard switches to the selected input method
    5. On the device's Settings, select Keyman as the input method
    6. On the Keyman keyboard, short-press and release the globe button
    7. Verify the previous input method is selected

  • TEST_SYSTEM_ROTATE_P-TO-L:

    1. With Keyman not active, load it in a portrait orientation.
    2. Once loaded, rotate the device to a landscape orientation.
      • On Android devices, you may need to press something like this for the rotation to occur:
      • The test "passes" if the OSK rotates properly.

  • TEST_SYSTEM_ROTATE_L-TO-P:

    1. With Keyman not active, load it in a landscape orientation.
    2. Once loaded, rotate the device to a portrait orientation.
      • On Android devices, you may need to press something like this for the rotation to occur:
      • The test "passes" if the OSK rotates properly.

  • TEST_EXT_BLUETOOTH_KEYBOARD: External (Bluetooth) keyboard
    This test is run with an external keyboard (USB or bluetooth) connected to the device

    Expand to see how to do and verify this
    1. Start with English(EuroLatin SIL) as the selected keyboard
    2. On the device's Settings, show the input method (OSK)
    3. Verify can type with numbers, letters, and punctuation
    4. Verify caps lock has no effect when typing
    5. Verify long-press on a character repeats that character
    6. Verify tab toggles to next field
    7. Verify enter adds a newline
    8. Verify backspace removes a character
    9. Verify ctrl+tab brings up the keyboard picker menu
    10. Use tab or arrow keys to navigate the keyboard picker menu
    11. Select a non-Latin keyboard
    12. Resume typing and verify new script appears

  • TEST_EXT_AZERTY_KEYBOARD: External European AZERTY keyboard
    This test is run with an external European AZERTY keyboard paired to the device. It should contain the 102nd key <> on the bottom row next to the left-shift. (bug(android): 102nd key on hardware keyboard does not give correct output on Android 5.0-8.0 #6875)

    1. Start Keyman and install the French Basic keyboard
    2. From the physical keyboard, type on the top letter row (azerty)
    3. Verify azerty appears
    4. Type on the 102nd key
    5. Verify < appears
    6. Type shift-102nd key
    7. Verify > appears

SUITE_KB_DISTRIBUTION: Ad-hoc Keyboard Distribution Tests

click to expand

Various versions of Androids

UI for these tests assume default English locale

  • GROUP_ANDROID_5
  • GROUP_ANDROID_8
  • GROUP_ANDROID_11

  • TEST_KMP_DISTRIBUTION KMP distribution

    1. Reset app settings (in Android Pie: Settings>Storage>Other apps>Keyman>Clear Storage)
    2. Verify kmp distribution successfully installs

      Example: Khmer Angkor
    3. Verify kmp successfully installs from Settings --> Install Keyboard or Dictionary --> Install from local file

SUITE_KNOWN_BUGS: drawn from a list of bugs

click to expand

Various versions of Androids

UI for these tests assume default English locale

  • GROUP_ANDROID_5
  • GROUP_ANDROID_8
  • GROUP_ANDROID_11
  • GROUP_ANDROID_12

  • TEST_GLOBE_KEY Holding backspace should not trigger a longpress on another random key (7155)

    1. In the Keyman app with the default sil_euro_latin keyboard and lexical model enabled, start typing a word
    2. Accept a suggestion
    3. Type another letter
    4. Hold the backspace key
    5. The previous letter is deleted
    6. Observe the longpress menu of the previous letter appearing
    7. FAIL this test if the longpress menu of the previous letter appears

  • TEST_FLICKER_BANNER unnecessary flicker on banner while typing (7162)

    1. In the Keyman app with the default sil_euro_latin keyboard and lexical model enabled, start typing "Fascinating"
    2. observe whether or not the suggestion banner is flickering on each keystroke.
    3. If flickering happens, FAIL this test.

  • TEST_SUGGESTION_NO_CAPITAL Initial suggestions when starting a new field are not always Initial cased (7165)

    1. In the Keyman app with the default sil_euro_latin keyboard and lexical model enabled, start typing some text (any text really)
    2. Clear the text using the option from the top right menu and observe the suggestion banner
    3. If the banner doesn't suggest words with capitalization, FAIL this test.

  • TEST_BROKEN_SUGGESTION selecting the "undo" suggestion breaks subsequent input (7167)

    When we select the "undo" suggestion that becomes available with Backspace after accepting a suggestion, on the next keystroke Keyman shouldn't inserts a broken suggestion.

    1. In Keyman for Android, type Prob.
    2. Select Problem from banner.
    3. Press Backspace, then select "Prob" (undoing the last suggestion).
    4. Then press Spacebar.
    5. If Prob is shown, PASS this test; otherwise if Prothe or Problem !

  • TEST_SHIFT_KEY_STUCK At start of input, pressing globe key, going back, leads to stuck shift key (7169)

  1. In Keyman for Android with the default keyboard on, press and hold the globe key to open the keyboard menu
  2. Press the < button on the top left of the screen or Android' back button at the bottom of the screen
  3. If the Shift key is stuck and unusable, FAIL this test.

  • TEST_REDUCED_TOAST_NOISE - Verifies the number of Toast notifications are reduced (7170)

    1. Install the PR build of Keyman for Android
    2. Launch the app and observe the "Get Started" menu is displayed
    3. Verify there's no Toast notification about checking the cloud catalog for updates
    4. Dismiss the "Get Started" menu and go to Keyman settings --> Installed Languages
    5. Click on "+" and search for "afade" language keyboard to install (install a keyboard that has no associated dictionary)
    6. Install sil_cameroon_azerty and install the keyboard for the "Afade" language"
    7. After the keyboard package is installed, verify no Toast notifications appear about checking for associated dictionary
    8. Go to Keyman Settings --> Installed Languages --> Afade
    9. On the Afade language settings menu, click the bottom "Dictionary - Check for available dictionary"
    10. Verify a Toast notification appears about checking for associated dictionary to download
    11. Wait a few minutes and verify no additional toast notifications appear
    12. Go to Keyman settings --> Installed Languages
    13. Click on "+" and search for khmer_angkor (install a keyboard that has an associated dictionary)
    14. Install khmer_angkor
    15. After the keyboard package is installed, verify:
    • no Toast notifications appear about "checking for associated dictionary"
    • two Toast notifications do appear about a "dictionary being installed in the background" and eventually "Resources successfully updated" (this may take a while depending on network lag)

  • TEST_PRESS_AND_HOLD_BACKSPACE Holding backspace appears to desync context between web and app. (7172)

    1. In Keyman app, type a few words without or without using the suggestion banner
    2. Press and hold the backspace
    3. If nothing gets deleted and subkeys on a random key appears, FAIL this test.

  • TEST_RAPID_TYPING Rapid typing on first caps letter sometimes gives TWo cap letters (7173)

    1. In Keyman app, while on Shift layer either in the middle or beginning of the sentence, type these three keys T​w​o rapidly and consecutively multiple times separated by a space
    2. If any of the output is TWo, FAIL this test.

    You can also try any other words that allow quick typing of two thumb, i.e. the first key is on the left thumb and the second on the right.
@MakaraSok MakaraSok added the test Any acceptance test issue label Aug 23, 2022
@MakaraSok MakaraSok added this to the A16S9 milestone Aug 23, 2022
@keymanapp-test-bot keymanapp-test-bot bot added has-user-test user-test-required User tests have not been completed labels Aug 23, 2022
@keymanapp-test-bot
Copy link

keymanapp-test-bot bot commented Aug 23, 2022 •
edited

User Test Results

Test specification and instructions

✅ SUITE_INSPECT: Visual Inspection of master branch

2 tests in 1 groups PASSED
  • ✅ TEST_CI_WEB_MASTER (PASSED): Verified the latest CI Android build of master successfully.
  • ✅ TEST_VERIFY_HISTORY (PASSED): Verified the History.md file and it contains all the current changes.

✅ SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

36 tests in 3 groups PASSED

  • ✅ GROUP_ANDROID_5:

    12 tests PASSED
    • ✅ TEST_GET_STARTED (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_INSTALL_KD (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_KEYMAN_SETTTINGS (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_SHARE (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_KEYMAN_BROWSER (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_TEXT_SIZE (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_CLEAR_TEXT (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_INFO (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_INSTALL_UPDATES (PASSED): Retested this in API 21 / Android 5.0 emulator and now I am able to see the Install Updates option menu in the menu lists. Able to upgraded it without having any problem.
    • ✅ TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_ADJUST_K_HEIGHT (PASSED): Tested this as per the instructions and it is working as expected.
    • ✅ TEST_SPACEBAR_CAPTION (PASSED): Tested this as per the instructions and it is working as expected.

  • ✅ GROUP_ANDROID_8:

    12 tests PASSED
    • ✅ TEST_GET_STARTED (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INSTALL_KD (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_KEYMAN_SETTTINGS (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SHARE (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_KEYMAN_BROWSER (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_TEXT_SIZE (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_CLEAR_TEXT (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INFO (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INSTALL_UPDATES (PASSED): Retested this in API 26 / Android 8.0 emulator and now I am able to see the Install Updates option menu in the menu lists. Able to upgraded it without having any problem.
    • ✅ TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_ADJUST_K_HEIGHT (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SPACEBAR_CAPTION (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.

  • ✅ GROUP_ANDROID_11:

    12 tests PASSED
    • ✅ TEST_GET_STARTED (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_INSTALL_KD (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_KEYMAN_SETTTINGS (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_SHARE (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_KEYMAN_BROWSER (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_TEXT_SIZE (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_CLEAR_TEXT (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_INFO (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_INSTALL_UPDATES (PASSED): Retested this in API 30 / Android 11.0 emulator and now I am able to see the Install Updates option menu in the menu lists. Able to upgraded it without having any problem.
    • ✅ TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_ADJUST_K_HEIGHT (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
    • ✅ TEST_SPACEBAR_CAPTION (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.

🟥 SUITE_KEYBOARD_FUNCTIONALITY: Keyboard Tests

  • 🟥 GROUP_ANDROID_5:

    • ✅ TEST_INAPP_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_LONG_PRESS (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_LONG_PRESS (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_KEYBOARD_PICKER (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • ✅ TEST_EXT_BLUETOOTH_KEYBOARD (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
    • 🟥 TEST_EXT_AZERTY_KEYBOARD (FAILED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is showing the wrong output for > and < keys. (notes)

  • 🟥 GROUP_ANDROID_8:

    • ✅ TEST_INAPP_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_LONG_PRESS (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_INAPP_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_LONG_PRESS (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_KEYBOARD_PICKER (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_SYSTEM_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • ✅ TEST_EXT_BLUETOOTH_KEYBOARD (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.
    • 🟥 TEST_EXT_AZERTY_KEYBOARD (FAILED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is showing the wrong output for > and < keys. (notes)

  • ✅ GROUP_ANDROID_11:

    14 tests PASSED
    • ✅ TEST_INAPP_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_INAPP_LONG_PRESS (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_INAPP_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_INAPP_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_SYSTEM_LONG_PRESS (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_KEYBOARD_PICKER (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_SYSTEM_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_SYSTEM_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_EXT_BLUETOOTH_KEYBOARD (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
    • ✅ TEST_EXT_AZERTY_KEYBOARD (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.

✅ SUITE_KB_DISTRIBUTION: Ad-hoc Keyboard Distribution Tests

3 tests in 3 groups PASSED

  • ✅ GROUP_ANDROID_5:

    1 tests PASSED
    • ✅ TEST_KMP_DISTRIBUTION (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.

  • ✅ GROUP_ANDROID_8:

    1 tests PASSED
    • ✅ TEST_KMP_DISTRIBUTION (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • ✅ GROUP_ANDROID_11:

    1 tests PASSED
    • ✅ TEST_KMP_DISTRIBUTION (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.

🟥 SUITE_KNOWN_BUGS: drawn from a list of bugs

  • 🟥 GROUP_ANDROID_5:

    • 🟥 TEST_GLOBE_KEY (FAILED): Tested this with API 21 / Android 5.0 emulator (Keyman 16.0.54-alpha build) and I observed that while holding backspace key, the long press menu of the previous letter appears on the view.
    • 🟥 TEST_FLICKER_BANNER (FAILED): Tested this with Keyman 16.0.55-alpha build in API 21 / Android 5.0 emulator and it seems that the flickering happens every time I press a key on the OSK. Seems to be an issue.
    • 🟥 TEST_SUGGESTION_NO_CAPITAL (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that after clearing the text in the tex pane, the banner suggest words without capitalization.
    • 🟥 TEST_BROKEN_SUGGESTION (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that the word 'Problem' appears on the screen after pressing the Spacebar. (notes)
    • ✅ TEST_SHIFT_KEY_STUCK (PASSED): Tested this with Keyman 16.0.60-alpha build in API 21 / Android 5 emulator and the Shift key is working as expected after returning from the language picker menu.
    • ✅ TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.60-alpha build in API 21 / Android 5 emulator and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation.
    • ✅ TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character.
    • 🟥 TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane. (notes)

  • 🟥 GROUP_ANDROID_8:

    • 🟥 TEST_GLOBE_KEY (FAILED): Tested this with API 26 / Android 8.0 emulator (Keyman 16.0.54-alpha build) and I observed that while holding backspace key, the long press menu of the previous letter appears on the view.
    • 🟥 TEST_FLICKER_BANNER (FAILED): Tested this with Keyman 16.0.55-alpha build in API 26 / Android 8.0 emulator and it seems that the flickering happens every time I press a key on the OSK. Seems to be an issue.
    • 🟥 TEST_SUGGESTION_NO_CAPITAL (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that after clearing the text in the text pane, the banner suggest words without capitalization. (notes)
    • 🟥 TEST_BROKEN_SUGGESTION (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that the word 'Problem' appears on the screen after pressing the Spacebar.
    • ✅ TEST_SHIFT_KEY_STUCK (PASSED): Tested this with Keyman 16.0.61-alpha build in API 26 / Android 8 emulator and the Shift key is working as expected after returning from the language picker menu.
    • ✅ TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.61-alpha build in API 26 / Android 8 emulator and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation.
    • ✅ TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character.
    • 🟥 TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane. (notes)

  • 🟥 GROUP_ANDROID_11:

    • 🟥 TEST_GLOBE_KEY (FAILED): Tested this with the Android Mobile device Version 11.0 (Keyman 16.0.54-alpha build) and I observed that while holding backspace key, the long press menu of the previous letter appears on the view. (notes)
    • 🟥 TEST_FLICKER_BANNER (FAILED): Tested this with Keyman 16.0.55-alpha build in the Android Mobile device Version 11.0 and it seems that the flickering happens every time I press a key on the OSK. Some time blank banner appears on the OSK. (notes)
    • 🟥 TEST_SUGGESTION_NO_CAPITAL (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that after clearing the text in the tex pane, the banner suggest words without capitalization.
    • 🟥 TEST_BROKEN_SUGGESTION (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that the word 'Problem' appears on the screen after pressing the Spacebar.
    • ✅ TEST_SHIFT_KEY_STUCK (PASSED): Tested this with Keyman 16.0.61-alpha build in the Android Mobile device Ver 11.0 and the Shift key is working as expected after returning from the language picker menu.
    • ✅ TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.61-alpha build in my Android Mobile device Version 11.0 and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation.
    • ✅ TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character.
    • 🟥 TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane. (notes)

  • 🟥 GROUP_ANDROID_12:

    • ✅ TEST_GLOBE_KEY (PASSED): no longpress menu appears when press and hold on the backspace
    • 🟥 TEST_FLICKER_BANNER (FAILED): the banner of the same suggested words flicks on the each keystroke and backspace
    • 🟥 TEST_SUGGESTION_NO_CAPITAL (FAILED): the suggestions are shown in small caps when the OSK has the Shift layer activated (see the screenshot below) (notes)
    • 🟥 TEST_BROKEN_SUGGESTION (FAILED): "Prob" does show as one of the suggestions (see the screenshot below) (notes)
    • ✅ TEST_SHIFT_KEY_STUCK (PASSED): the Shift key is unstuck and usable as usual
    • ✅ TEST_REDUCED_TOAST_NOISE (PASSED): the toast about checking for associated dictionary to download doesn't show up as indicated
    • ✅ TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): everything gets delete when press and hold on the backspace and no random key appears either
    • 🟥 TEST_RAPID_TYPING (FAILED): type T W O rapidly outputs "TWo" at times (tested on a real device, Android 10, SG9+
Retesting Template 
@keymanapp-test-bot retest SUITE_KEYBOARD_FUNCTIONALITY GROUP_ANDROID_5 TEST_EXT_AZERTY_KEYBOARD GROUP_ANDROID_8 TEST_EXT_AZERTY_KEYBOARD SUITE_KNOWN_BUGS GROUP_ANDROID_5 TEST_GLOBE_KEY TEST_FLICKER_BANNER TEST_SUGGESTION_NO_CAPITAL TEST_BROKEN_SUGGESTION TEST_RAPID_TYPING GROUP_ANDROID_8 TEST_GLOBE_KEY TEST_FLICKER_BANNER TEST_SUGGESTION_NO_CAPITAL TEST_BROKEN_SUGGESTION TEST_RAPID_TYPING GROUP_ANDROID_11 TEST_GLOBE_KEY TEST_FLICKER_BANNER TEST_SUGGESTION_NO_CAPITAL TEST_BROKEN_SUGGESTION TEST_RAPID_TYPING GROUP_ANDROID_12 TEST_FLICKER_BANNER TEST_SUGGESTION_NO_CAPITAL TEST_BROKEN_SUGGESTION TEST_RAPID_TYPING

@bharanidharanj
Copy link

SUITE_INSPECT: Visual Inspection of master branch

  • TEST_CI_WEB_MASTER (PASSED): Verified the latest CI Android build of master successfully.
  • TEST_VERIFY_HISTORY (PASSED): Verified the History.md file and it contains all the current changes.

@bharanidharanj
Copy link

bharanidharanj commented Aug 23, 2022 •
edited

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

  • TEST_GET_STARTED (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_INSTALL_KD (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_KEYMAN_SETTTINGS (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_SHARE (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_KEYMAN_BROWSER (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_TEXT_SIZE (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_CLEAR_TEXT (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_INFO (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_INSTALL_UPDATES (SKIPPED): notes
  • TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_ADJUST_K_HEIGHT (PASSED): Tested this as per the instructions and it is working as expected.
  • TEST_SPACEBAR_CAPTION (PASSED): Tested this as per the instructions and it is working as expected.

@bharanidharanj
Copy link

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_11:

  • TEST_GET_STARTED (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_INSTALL_KD (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_KEYMAN_SETTTINGS (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_SHARE (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_KEYMAN_BROWSER (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_TEXT_SIZE (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_CLEAR_TEXT (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_INFO (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_INSTALL_UPDATES (SKIPPED): notes
  • TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_ADJUST_K_HEIGHT (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.
  • TEST_SPACEBAR_CAPTION (PASSED): Tested this in my Android Mobile device Version 11.0 as per the instructions and it is working as expected.

@bharanidharanj
Copy link

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_8:

  • TEST_GET_STARTED (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_INSTALL_KD (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_KEYMAN_SETTTINGS (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_SHARE (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_KEYMAN_BROWSER (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_TEXT_SIZE (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_CLEAR_TEXT (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_INFO (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_INSTALL_UPDATES (SKIPPED): notes
  • TEST_CHANGE_DISPLAY_LANG (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_ADJUST_K_HEIGHT (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.
  • TEST_SPACEBAR_CAPTION (PASSED): Tested this in API 26 / Android 8.0 emulator and it is working as expected.

@bharanidharanj
Copy link

bharanidharanj commented Aug 24, 2022 •
edited

SUITE_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_8:

  • TEST_INAPP_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_INAPP_LONG_PRESS (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_INAPP_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_INAPP_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_SYSTEM_LONG_PRESS (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_KEYBOARD_PICKER (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_SYSTEM_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_SYSTEM_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_EXT_BLUETOOTH_KEYBOARD (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

  • TEST_EXT_AZERTY_KEYBOARD (FAILED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is showing the wrong output for > and < keys.

@bharanidharanj
Copy link

SUITE_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_5:

  • TEST_INAPP_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_INAPP_LONG_PRESS (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_INAPP_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_INAPP_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_SYSTEM_LONG_PRESS (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_KEYBOARD_PICKER (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_SYSTEM_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_SYSTEM_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_EXT_BLUETOOTH_KEYBOARD (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.
  • TEST_EXT_AZERTY_KEYBOARD (FAILED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is showing the wrong output for > and < keys.

@bharanidharanj
Copy link

SUITE_KEYBOARD_FUNCTIONALITY: Keyboard Tests

GROUP_ANDROID_11:

  • TEST_INAPP_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_INAPP_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_INAPP_LONG_PRESS (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_INAPP_NON_LATIN_KEYBOARD (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_INAPP_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_INAPP_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_SYSTEM_LATIN_KEYBOARD_PORTRAIT (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_SYSTEM_LATIN_KEYBOARD_LANDSCAPE (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_SYSTEM_LONG_PRESS (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_KEYBOARD_PICKER (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_SYSTEM_ROTATE_P-TO-L (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_SYSTEM_ROTATE_L-TO-P (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_EXT_BLUETOOTH_KEYBOARD (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.
  • TEST_EXT_AZERTY_KEYBOARD (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.

@bharanidharanj
Copy link

SUITE_KB_DISTRIBUTION: Ad-hoc Keyboard Distribution Tests

GROUP_ANDROID_5:

  • TEST_KMP_DISTRIBUTION (PASSED): Tested this as per the instructions in API 21 / Android 5.0 emulator and it is working as expected.

GROUP_ANDROID_8:

  • TEST_KMP_DISTRIBUTION (PASSED): Tested this as per the instructions in API 26 / Android 8.0 emulator and it is working as expected.

GROUP_ANDROID_11:

  • TEST_KMP_DISTRIBUTION (PASSED): Tested this as per the instructions in my Android Mobile device Version 11.0 and it is working as expected.

@bharanidharanj
Copy link

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

  • TEST_INSTALL_UPDATES (FAILED): Retested this in API 21 / Android 5.0 emulator and it seems that the 'Install Updates' menu option is not available in the Keyman menu.

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_8:

  • TEST_INSTALL_UPDATES (FAILED): Retested this in API 26 / Android 8.0 emulator and it seems that the 'Install Updates' menu option is not available in the Keyman menu.

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_11:

  • TEST_INSTALL_UPDATES (FAILED): Retested this in my Android Mobile Version 11.0 and it seems that the 'Install Updates' menu option is not available in the Keyman menu.

@keymanapp-test-bot keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Aug 24, 2022
@bharanidharanj
Copy link

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_5:

  • TEST_INSTALL_UPDATES (PASSED): Retested this in API 21 / Android 5.0 emulator and now I am able to see the Install Updates option menu in the menu lists. Able to upgraded it without having any problem.

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_8:

  • TEST_INSTALL_UPDATES (PASSED): Retested this in API 26 / Android 8.0 emulator and now I am able to see the Install Updates option menu in the menu lists. Able to upgraded it without having any problem.

SUITE_MENU_FUNCTIONALITY: Menu Functionality Tests

GROUP_ANDROID_11:

  • TEST_INSTALL_UPDATES (PASSED): Retested this in API 30 / Android 11.0 emulator and now I am able to see the Install Updates option menu in the menu lists. Able to upgraded it without having any problem.

@keymanapp-test-bot keymanapp-test-bot bot added the user-test-required User tests have not been completed label Sep 2, 2022
@bharanidharanj
Copy link

bharanidharanj commented Sep 2, 2022 •
edited

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_5:

  • TEST_GLOBE_KEY (FAILED): Tested this with API 21 / Android 5.0 emulator (Keyman 16.0.54-alpha build) and I observed that while holding backspace key, the long press menu of the previous letter appears on the view.

  • TEST_FLICKER_BANNER (PASSED): Tested this with Keyman 16.0.54-alpha build and I observed that flickering did not happen on each keystroke on the suggestion banner.

  • TEST_SUGGESTION_NO_CAPITAL (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that after clearing the text in the tex pane, the banner suggest words without capitalization.

  • TEST_BROKEN_SUGGESTION (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that the word 'Problem' appears on the screen after pressing the Spacebar.

GROUP_ANDROID_8:

  • TEST_GLOBE_KEY (FAILED): Tested this with API 26 / Android 8.0 emulator (Keyman 16.0.54-alpha build) and I observed that while holding backspace key, the long press menu of the previous letter appears on the view.

  • TEST_FLICKER_BANNER (PASSED): Tested this with Keyman 16.0.54-alpha build and I observed that flickering did not happen on each keystroke on the suggestion banner.

  • TEST_SUGGESTION_NO_CAPITAL (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that after clearing the text in the text pane, the banner suggest words without capitalization.

  • TEST_BROKEN_SUGGESTION (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that the word 'Problem' appears on the screen after pressing the Spacebar.

GROUP_ANDROID_11:

  • TEST_GLOBE_KEY (FAILED): Tested this with the Android Mobile device Version 11.0 (Keyman 16.0.54-alpha build) and I observed that while holding backspace key, the long press menu of the previous letter appears on the view.

  • TEST_FLICKER_BANNER (PASSED): Tested this with Keyman 16.0.54-alpha build and I observed that flickering did not happen on each keystroke on the suggestion banner.

  • TEST_SUGGESTION_NO_CAPITAL (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that after clearing the text in the tex pane, the banner suggest words without capitalization.

  • TEST_BROKEN_SUGGESTION (FAILED): Tested this with Keyman 16.0.54-alpha build and I noticed that the word 'Problem' appears on the screen after pressing the Spacebar.

@keymanapp-test-bot keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Sep 2, 2022
@mcdurdin
Copy link
Member

mcdurdin commented Sep 2, 2022

In terms of:

TEST_FLICKER_BANNER (PASSED)

This has been observed with current master and stable builds. So we need to make sure that we are able to reproduce the issue in this regression test. The flickering is that the set of suggestions disappears and reappears without the actual suggestions changing, or when the suggestions change, there is a brief moment when no suggestions are visible on the banner. To pass this test, the suggestions would change from one set to another and at no time would there be any blank banner.

@keymanapp-test-bot retest TEST_FLICKER_BANNER

@keymanapp-test-bot keymanapp-test-bot bot added the user-test-required User tests have not been completed label Sep 2, 2022
@mcdurdin mcdurdin modified the milestones: A16S9, A16S10 Sep 5, 2022
@bharanidharanj
Copy link

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_5:

  • TEST_FLICKER_BANNER (FAILED): Tested this with Keyman 16.0.55-alpha build in API 21 / Android 5.0 emulator and it seems that the flickering happens every time I press a key on the OSK. Seems to be an issue.

GROUP_ANDROID_8:

  • TEST_FLICKER_BANNER (FAILED): Tested this with Keyman 16.0.55-alpha build in API 26 / Android 8.0 emulator and it seems that the flickering happens every time I press a key on the OSK. Seems to be an issue.

GROUP_ANDROID_11:

  • TEST_FLICKER_BANNER (FAILED): Tested this with Keyman 16.0.55-alpha build in the Android Mobile device Version 11.0 and it seems that the flickering happens every time I press a key on the OSK. Some time blank banner appears on the OSK.

@keymanapp-test-bot keymanapp-test-bot bot added user-test-required User tests have not been completed and removed user-test-required User tests have not been completed labels Sep 5, 2022
@bharanidharanj
Copy link

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_5:

  • TEST_SHIFT_KEY_STUCK (PASSED): Tested this with Keyman 16.0.60-alpha build in API 21 / Android 5 emulator and the Shift key is working as expected after returning from the language picker menu.

@bharanidharanj
Copy link

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_5:

  • TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.60-alpha build in API 21 / Android 5 emulator and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation.

  • TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character.

  • TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane.

@mcdurdin
Copy link
Member

mcdurdin commented Sep 13, 2022 via email

@bharanidharanj
Copy link

Can you test these with 15.0.270 also please? We just released that version last night.
…
On Tue, 13 Sept 2022, 6:04 pm bharanidharanj, @.> wrote: SUITE_KNOWN_BUGS: drawn from a list of bugs <#7161> GROUP_ANDROID_5: - TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.60-alpha build in API 21 / Android 5 emulator and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation. - TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character. - TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane. https://user-images.githubusercontent.com/19683143/189845790-898614b9-f3df-434f-a6d8-e93a5472d59c.PNG — Reply to this email directly, view it on GitHub <#7099 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCKHPOKFGBYWYUGXJCGGKLV6AYQXANCNFSM57JZXBDA . You are receiving this because you commented.Message ID: @.>

@mcdurdin Sure. I will do it.

@bharanidharanj
Copy link

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_8:

  • TEST_SHIFT_KEY_STUCK (PASSED): Tested this with Keyman 16.0.61-alpha build in API 26 / Android 8 emulator and the Shift key is working as expected after returning from the language picker menu.

  • TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.61-alpha build in API 26 / Android 8 emulator and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation.

  • TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character.

  • TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane.

@bharanidharanj
Copy link

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_11:

  • TEST_SHIFT_KEY_STUCK (PASSED): Tested this with Keyman 16.0.61-alpha build in the Android Mobile device Ver 11.0 and the Shift key is working as expected after returning from the language picker menu.
  • TEST_REDUCED_TOAST_NOISE (PASSED): Tested with Keyman 16.0.61-alpha build in my Android Mobile device Version 11.0 and verified a Toast notification appears about checking for associated dictionary to download after clicking the bottom menu option. Also, verified that no Toast notifications appears about 'checking for associated dictionary'. Also, noticed that 2 Toast notifications appears during the installation.
  • TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): Tested this as per the instructions and verified that the holding backspace key deletes all characters one by one including the last character.
  • TEST_RAPID_TYPING (FAILED): Tested this as per the instructions and yes it produces two cap letters while rapidly typing on the text pane.

@keymanapp-test-bot keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Sep 13, 2022
@bharanidharanj
Copy link

bharanidharanj commented Sep 13, 2022 •
edited

@mcdurdin Tested Suite_Known_Bugs with Keyman 15.0.270 Stable build in API 21 / Android 5 emulator and here are the results:
Test_Shift_Key_Stuck (PASSED)
Test_Reduced_Toast_Noise (FAILED)
Test_Press_And_Hold_Backspace (PASSED)
Test_Rapid_Typing (FAILED)

@bharanidharanj
Copy link

Tested Suite_Known_Bugs with Keyman 15.0.270 Stable build in API 26 / Android 8 and here are the results:
Test_Shift_Key_Stuck (PASSED)
Test_Reduced_Toast_Noise (FAILED)
Test_Press_And_Hold_Backspace (PASSED)
Test_Rapid_Typing (FAILED)

@bharanidharanj
Copy link

Tested Suite_Known_Bugs with Keyman 15.0.270 Stable build in Android Mobile device version 11.0 and here are the results:
Test_Shift_Key_Stuck (PASSED)
Test_Reduced_Toast_Noise (FAILED)
Test_Press_And_Hold_Backspace (PASSED)
Test_Rapid_Typing (FAILED)

@mcdurdin
Copy link
Member

@bharanidharanj thank you -- those test results for 15.0.270 match up with what I was hoping for!

@mcdurdin mcdurdin modified the milestones: A16S10, A16S11 Sep 17, 2022
@mcdurdin mcdurdin modified the milestones: A16S11, A16S12 Oct 2, 2022
@keymanapp-test-bot keymanapp-test-bot bot added the user-test-required User tests have not been completed label Oct 5, 2022
@MakaraSok
Copy link
Collaborator Author

MakaraSok commented Oct 5, 2022 •
edited

Test Results

Tested with Keyman's latest alpha 16.0.74 on Android 12.0 - Pixel 5 API 31

SUITE_KNOWN_BUGS: drawn from a list of bugs

GROUP_ANDROID_12:

  • TEST_GLOBE_KEY (PASSED): no longpress menu appears when press and hold on the backspace

  • TEST_FLICKER_BANNER (FAILED): the banner of the same suggested words flicks on the each keystroke and backspace

  • TEST_SUGGESTION_NO_CAPITAL (FAILED): the suggestions are shown in small caps when the OSK has the Shift layer activated (see the screenshot below)

    image

  • TEST_BROKEN_SUGGESTION (FAILED): "Prob" does show as one of the suggestions (see the screenshot below)

    image

  • TEST_SHIFT_KEY_STUCK (PASSED): the Shift key is unstuck and usable as usual

  • TEST_REDUCED_TOAST_NOISE (PASSED): the toast about checking for associated dictionary to download doesn't show up as indicated

  • TEST_PRESS_AND_HOLD_BACKSPACE (PASSED): everything gets delete when press and hold on the backspace and no random key appears either

  • TEST_RAPID_TYPING (FAILED): type T W O rapidly outputs "TWo" at times (tested on a real device, Android 10, SG9+

@keymanapp-test-bot keymanapp-test-bot bot removed the user-test-required User tests have not been completed label Oct 6, 2022
@mcdurdin mcdurdin modified the milestones: A16S12, A16S13 Oct 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
android/ has-user-test test Any acceptance test issue user-test-failed
Milestone
A16S13
Development

No branches or pull requests
3 participants
@mcdurdin @bharanidharanj @MakaraSok