-
-
Notifications
You must be signed in to change notification settings - Fork 103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(web): Regression Test for KMW Alpha 16.0.29 #6923
Comments
User Test ResultsTest specification and instructions ✅ SUITE_INSPECT: Visual Inspection of 16.0.29 Alpha4 tests in 1 groups PASSED
✅ SUITE_DESKTOP: Desktop Platforms108 tests in 6 groups PASSED
✅ SUITE_TOUCH: Touch platform tests60 tests in 6 groups PASSED
|
SUITE_INSPECT: Visual Inspection of 16.0.29 Alpha
|
SUITE_DESKTOP: Desktop PlatformsGROUP_WINDOWS_FIREFOX:
|
SUITE_DESKTOP: Desktop PlatformsGROUP_LINUX_FIREFOX:
|
SUITE_TOUCH: Touch platform testsGROUP_ANDROID_CHROME_TOUCH:
|
The Swedish test was failing because we had a deployment issue with version 2.0 of sil_euro_latin, which I have since resolved on the backend. @keymanapp-test-bot retest SUITE_DESKTOP GROUP_WINDOWS_FIREFOX TEST_BASELINE_SWEDISH GROUP_WINDOWS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_SAFARI TEST_BASELINE_SWEDISH GROUP_MACOS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_FIREFOX TEST_BASELINE_SWEDISH |
SUITE_DESKTOP: Desktop PlatformsGROUP_WINDOWS_FIREFOX:
GROUP_WINDOWS_CHROME:
GROUP_MACOS_CHROME:
GROUP_MACOS_FIREFOX:
GROUP_MACOS_SAFARI:
|
SUITE_TOUCH: Touch platform testsGROUP_IPHONE_SAFARI_TOUCH:
GROUP_IPHONE_SAFARI_HARDWARE:
|
SUITE_TOUCH: Touch platform testsGROUP_ANDROID_CHROME_HARDWARE:
|
SUITE_TOUCH: Touch platform testsGROUP_IPAD_SAFARI_TOUCH:
GROUP_IPAD_SAFARI_HARDWARE:
|
SUITE_DESKTOP: Desktop PlatformsGROUP_WINDOWS_FIREFOX:
|
SUITE_DESKTOP: Desktop PlatformsGROUP_WINDOWS_CHROME:
|
SUITE_DESKTOP: Desktop PlatformsGROUP_MACOS_SAFARI:
GROUP_MACOS_CHROME:
GROUP_MACOS_FIREFOX:
|
SUITE_DESKTOP: Desktop PlatformsGROUP_LINUX_FIREFOX:
|
@keymanapp-test-bot retest SUITE_DESKTOP GROUP_WINDOWS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_SAFARI TEST_BASELINE_SWEDISH GROUP_MACOS_CHROME TEST_BASELINE_SWEDISH @bharanidharanj I cannot replicate the behavior. Let's retest these in a rubber ducky session. |
SUITE_DESKTOP: Desktop PlatformsGROUP_WINDOWS_CHROME:
GROUP_MACOS_SAFARI:
GROUP_MACOS_CHROME:
|
@jahorton @mcdurdin Makara and I have retested the Swedish Keyboard issue in my Windows OS as well as in the Mac Mini M1 OS and we were able to reproduce it. But Makara was not able to reproduce this issue in his Virtual Machine. I am using Native Machine. (Windows and MacMini). So, we are confused whether the Native machine makes this kind of problem? |
I have flushed this file from the CDN cache to try and ensure there is nothing wrong there. Can you try clearing your browser cache and retrying these tests? @keymanapp-test-bot retest SUITE_DESKTOP GROUP_WINDOWS_CHROME TEST_BASELINE_SWEDISH GROUP_MACOS_SAFARI TEST_BASELINE_SWEDISH GROUP_MACOS_CHROME TEST_BASELINE_SWEDISH |
@mcdurdin Okay, I will retest it as per your instructions. |
SUITE_DESKTOP: Desktop PlatformsGROUP_WINDOWS_CHROME:
GROUP_MACOS_SAFARI:
GROUP_MACOS_CHROME:
|
Looks like everything in KMW worked well; sad there was some odd cache / s.keyman.com behavior that interfered at points, though. |
The cache issue with s.keyman.com relates to failed deployments we have been experiencing and for which I have a patch in the pipeline |
Keyman for Web Acceptance Test Procedures
Remember that KeymanWeb is embedded into the iOS and Android apps, thus we need to ensure that those work properly with a KeymanWeb release as well. See the Android and iOS instructions for details.
User Testing
SUITE_INSPECT: Visual Inspection of 16.0.29 Alpha
HISTORY.md
contains information on all the current changes.SUITE_DESKTOP: Desktop Platforms
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
Use the
web/samples/minified.html
sample page for the following tests.TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.ŋ
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ឞ
. Delete the character again.ស្រុក
.ស្រុ
ស្រ
ស្
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.Text Selection Tests [SKIP THIS FOR NOW AS THE KEYBOARD IS NOT SET UP FOR YET]
@mcdurdin Please at the following keyboard for this test. Thanks.
These tests are all run with the web_context_tests keyboard, in the Unminified Test Page. They should be run on both INPUT and TEXTAREA elements. Press Ctrl+F5 to reload the page between tests.
TEST_SELECTION: Type abcx. Select
bc
and type q. The output after the final key should beaqx
. (This is testing selection replacement with no matching keyboard rule.)TEST_SELECTION_2: Type abcx. Select
bc
and type p. The output after the final key should beaqx
. (This is testing selection replacement with a matching keyboard rule.)TEST_CONTEXT_BASE: Type abcd. The output after the final key should be
!
.TEST_CONTEXT_SELECTION: Type abcx. Select the
x
character, and type d. The output after the final key should beabcd
.TEST_CONTEXT_SELECTION_2: Type abcx. Select the
x
character, and delete it with Backspace. Type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_3: Type abcx. Select the
x
character, and type y. Press Backspace, and type d. The output after the final key should be!
.TEST_CONTEXT_SELECTION_4: Type xabcx. Select the
abc
characters, and type d. The output after the final key should bexdx
.Element Attachment Tests
web/testing/attachment-api
samples page to test the following:'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the
Tests the new Attachment/Enablement API functionality
test page.TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.Tests CJK keyboard functionality
Use the standard "Test unminified KeymanWeb" page.
TEST_JAPANESE_TYPING: Verify that a CJK keyboard still works.
japanese
, then click Add.a
. A "picker" displaying a few options should display:2
. The second option should replace the context.TEST_JAPANESE_FOCUS: Verify that a CJK keyboard is shown and hidden properly without interfering with a page's UX.
_TYPING
test).SUITE_TOUCH: Touch platform tests
Ensure that all tests in the selections below are run on a representative set of the following browsers and setups:
Baseline Tests
Use the
web/samples/minified.html
sample page for the following tests.TEST_BASELINE_ADD_SIL_IPA: Using the "Add a keyboard by keyboard name" section, add
sil_ipa
.ŋ
.TEST_BASELINE_ADD_KM: Using the "Add a keyboard by BCP-47 language code" section, add
km
. A Khmer keyboard should be added (perhaps khmer_basic, but not necessarily).TEST_BASELINE_ADD_KHMER_ANGKOR: Using the "Add a keyboard by keyboard name" section, add
khmer_angkor
.ឞ
. Delete the character again.ស្រុក
.ស្រុក
.ស្រុ
ស្រ
ស
TEST_BASELINE_ADD_SPANISH: Using the "Add a keyboard by language name(s) section", add
spanish
. Verify that a Spanish keyboard is added to the list.TEST_BASELINE_SWEDISH: Swap to the Swedish keyboard.
é
and that o \ e producesœ
.TEST_BASELINE_CAMEROON: If using a touch-based keyboard, add the following keyboard by name: "sil_cameroon_qwerty"
Element Attachment Tests
web/testing/attachment-api
samples page to test the following:'auto'
attachment mode properly and automatically attaches to each dynamically-added control.'manual'
attachment mode results in a page with no KMW-activated controls.Site-based tests of attachment API
All behavior exhibited here should feel "normal". These tests are solely to ensure that nothing got accidentally broken.
For each of these, make use of the
Tests the new Attachment/Enablement API functionality
test page.TEST_NORMAL_USE: Ensure that the keyboard displays and works correctly when first loaded up.
TEST_ELEMENT_HOPPING:
TEST_SPECIFIC_KEYBOARDS:
Dynamic area #0.
Dynamic area #1
's "Set to Dzongkha button.Dynamic area #1
. The Dzongkha keyboard should be displayed.Dynamic area #2
.Dynamic area #0.
The French keyboard should be displayed.Dynamic area #2.
The Lao keyboard should be displayed.Dynamic area #1.
The Dzongkha keyboard should be displayed.Dynamic area #0
's "Clear Keyboard" button.Dynamic area #0.
The Lao keyboard should be displayed.The text was updated successfully, but these errors were encountered: