Fix caps, keyboard group logic #39

tvquizphd · 2021-01-03T06:59:50Z

Update: For the latest status, see my latest comment.

This closes issue #37, closes issue #41, and ignores issue #46.

This addresses a retext-spell issue I've discovered that is fundamentally an nspell issue.

Addressing a bigger problem

In main, the notion of "Keyboard groups" doesn't really make sense as it replaces at a position (of a character in group) that is unrelated to the original position (of a character in word). This solution attempts to address that "root problem."

~~For another solution that ignores issue #41 but solves issue #37, see my alternative PR.~~

This solution splits the word at the original index of the character in the input word instead of the unrelated position of the character in the keyboard group. See my review comments.

It's ready to merge!

tvquizphd

This change has the effect that only upper-case replacements are suggested to replace upper-case characters, and only lower-case replacements are suggested to replace lower-case characters.

tvquizphd · 2021-01-03T07:13:42Z

lib/suggest.js

@@ -77,6 +77,8 @@ function suggest(value) {

  while (++index < length) {
    character = value.charAt(index)
+    before = value.slice(0, index)
+    after = value.slice(index + 1)


Here we slice the word at the index of the character in the input word. From my perspective, this more accurately simulates keyboard errors than the current approach.

tvquizphd · 2021-01-03T07:15:02Z

lib/suggest.js

@@ -90,8 +92,6 @@ function suggest(value) {
        continue
      }

-      before = value.slice(0, position)
-      after = value.slice(position + 1)


Why does the current approach slice the word at the position of the character in an unrelated group string such as the keyboard group "qwertzuop"?

wooorm · 2021-01-08T07:53:40Z

Nice! I think the new suggestions ([ 'Brandi', 'Shandy', 'handy', 'Ghana', 'Grand', 'Grandee', 'Grands', 'Shanxi', 'hand', 'hands' ] for Ghandi) might be better than what was suggested before btw, I’m fine with changing that one

tvquizphd · 2021-01-09T05:58:03Z

Update: The PR on my own fork has since been merged.

I've reworked this PR to be more efficient, differ less from main, and keep the exact behavior of the tests @wooorm looked at yesterday. Those changes have been committed to this PR, but there are a couple changes I'm hesitant to commit.

To frame the decision that remains: I have opened a PR against this PR on my own fork. It should be very quick to look at the 4 lines that PR changes in order to get a sense of why I believe that PR makes more logical sense, but will cause more breaking changes for anyone who relies on the exact ordering of the results.

Practical results

I've compared this PR against the PR on my own fork using a small test suite of single-letter errors in all 5-letter English words starting with T and ending with e. You can see the comparison here with this PR on the left and the PR on my own fork on the right.

In general, the PR on my own fork tends to favor shorter suggestions (made by removals) unless there are more specific suggestions that are actually based on keyboard groups instead of more or less at random. Here are some examples of how the top suggestions differ:

Given "Tepex" my own fork suggests "Telex" instead of this PR's "Tepee"
- because "x" and "e" are not in the same keyboard group.
Given "Thanj" my own fork suggests "Thanh" instead of this PR's "Thane"
- because "j" and "h" are both in the keyboard group "jhn".
Given "Threa" my own fork suggests "Threw" instead of this PR's "Three"
- because "a" and "w" are both in the keyboard group "qaw".
Given "Threk" my own fork suggests "Shrek" instead of this PR's "Three"
- because "k" and "e" are not in same keyboard group.
Given "Tildy" my own fork suggests "Tidy" instead of this PR's "Tilde"
- because "y" and "e" are not in the same keyboard group.

Odd observation

For all the words I've tested, this PR's top suggestion always ends in "e". That's explained by the fact that the fifth letter in each word will take replacements based on the "t" in "qwertzuop", which includes an "e". It's notable that "e" is the most common letter, but this benefit only befalls the fifth letter in words containing a t!

In essence, without the two changes on my own fork, the status quo in words with a "t" is to always prefer an "e" for the fifth letter... because "T" is the fifth key on the keyboard. It's madness, but it happens to work okay.

wooorm · 2021-01-09T11:43:48Z

Nice, thanks for all the research!

Your extra changes on the PR to the PR also look good. I do wonder how much they’d change the tests: from your explanation in the comment above I assume they’d make more sense, in which case, I’m 👍 on it

…yboard edits

test/index.js

lib/suggest.js

test/index.js

tvquizphd · 2021-01-10T10:54:54Z

I do wonder how much they’d change the tests: from your explanation in the comment above I assume they’d make more sense, in which case, I’m 👍 on it

There are essentially two new options for how the "Ghandi" case should be handled:

Look at this PR:
- ['Brandi', 'Ghana', 'Grandee', 'Shanxi', 'hand', 'hands', 'handy']
- Three results are based on a lowercase truncation of "Ghandi" to "handi".
Look at PR Fix keyboard edits, fix capitalization, and update tests #45:
- ['Brandi', 'Ghana', 'Grandee', 'Hand', 'Hands', 'Handy', 'Hanoi', 'Hindi', 'Randi', 'Shanxi']
- Four results depend on the uppercase truncation of "Ghandi" to "Handi".

For the difference between the two options, see issue #46.

tvquizphd · 2021-01-10T21:37:20Z

I'd just like to clarify that this PR is ready to merge.

tvquizphd · 2021-01-15T00:59:43Z

I realize the new issue #46 is off-topic for this PR, so I've reverted changes that address that issue. To make this PR easier to follow, I've collapsed any off-topic/outdated comments I've made. This PR now only addresses issues #37 and #41, as originally intended.

Now it's easiest to understand this PR by looking at the files changed. Despite all the commits I've added to this PR, it only alters lib/suggest by +11/-2 lines and test/index by +13/-7 lines.

wooorm · 2021-01-17T09:54:57Z

Awesome, thanks!

wooorm · 2021-01-17T14:49:43Z

released in 2.1.5!

tvquizphd marked this pull request as draft January 3, 2021 07:11

tvquizphd marked this pull request as ready for review January 3, 2021 07:12

tvquizphd commented Jan 3, 2021

View reviewed changes

This comment has been minimized.

Sign in to view

tvquizphd changed the title ~~Split word character index rather than group position~~ Split word at character index rather than group position Jan 3, 2021

tvquizphd marked this pull request as draft January 3, 2021 17:20

tvquizphd marked this pull request as ready for review January 6, 2021 06:17

tvquizphd marked this pull request as draft January 6, 2021 06:21

This comment has been minimized.

Sign in to view

tvquizphd mentioned this pull request Jan 8, 2021

Broken keyboard groups logic for initial edits #41

Closed

tvquizphd force-pushed the fix_extra_uppercase_root_problem branch 2 times, most recently from 3a26271 to bb64a97 Compare January 9, 2021 05:10

tvquizphd mentioned this pull request Jan 9, 2021

Reduce backwards compatibility in favor of more logical edits tvquizphd/nspell#1

Merged

10 tasks

tvquizphd mentioned this pull request Jan 9, 2021

No longer return all-caps for contractions with missing apostrophes #40

Merged

tvquizphd marked this pull request as ready for review January 9, 2021 06:26

tvquizphd added 2 commits January 9, 2021 14:15

Split at character index rather than group position when inserting ke…

ca06a2f

…yboard edits

Disallow duplicate replacements for each index

1d81ec1

tvquizphd force-pushed the fix_extra_uppercase_root_problem branch from 160efa2 to f16a3ee Compare January 9, 2021 19:15

tvquizphd added 3 commits January 9, 2021 14:18

replicate original keyboard group limitations

db2200c

ignore original keyboard group limitations

49bea85

update existing tests to reflect new behavior

cf45c5c

tvquizphd force-pushed the fix_extra_uppercase_root_problem branch 3 times, most recently from f4469c2 to 13ba741 Compare January 10, 2021 02:39

tvquizphd commented Jan 10, 2021

View reviewed changes

test/index.js Outdated Show resolved Hide resolved

test/index.js Outdated Show resolved Hide resolved

tvquizphd commented Jan 10, 2021

View reviewed changes

test/index.js Outdated Show resolved Hide resolved

tvquizphd force-pushed the fix_extra_uppercase_root_problem branch 2 times, most recently from 81c0f1b to 7d6a8de Compare January 10, 2021 07:28

tvquizphd added 2 commits January 10, 2021 02:42

Make capitalization more consistent and add new capitalization tests

2a0f3b7

update contraction test to reflect new behavior

830d803

tvquizphd force-pushed the fix_extra_uppercase_root_problem branch from 7d6a8de to 830d803 Compare January 10, 2021 07:42

tvquizphd commented Jan 10, 2021

View reviewed changes

test/index.js Outdated Show resolved Hide resolved

tvquizphd commented Jan 10, 2021

View reviewed changes

lib/suggest.js Outdated Show resolved Hide resolved

tvquizphd commented Jan 10, 2021

View reviewed changes

lib/suggest.js Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

adjust behavior for funky case when sentence-case in dictionary

c31a1cb

tvquizphd commented Jan 10, 2021

View reviewed changes

test/index.js Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

This was referenced Jan 10, 2021

Fix keyboard edits #42

Closed

Fix capitalization #43

Closed

Fix capitalization tests #44

Closed

Fix keyboard edits, fix capitalization, and update tests #45

Merged

This comment has been minimized.

Sign in to view

tvquizphd mentioned this pull request Jan 10, 2021

Output case should follow more consistent rules #46

Closed

10 tasks

Simplify this PR by reverting all additional capitalization fixes

ab0f1f8

This comment has been minimized.

Sign in to view

add more sensible test to ensure 100% coverage

8bf0fa9

wooorm merged commit 9c002d2 into wooorm:main Jan 17, 2021

wooorm changed the title ~~Split word at character index rather than group position~~ Fix caps, keyboard group logic Jan 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix caps, keyboard group logic #39

Fix caps, keyboard group logic #39

tvquizphd commented Jan 3, 2021 •

edited

Loading

tvquizphd left a comment

tvquizphd Jan 3, 2021

tvquizphd Jan 3, 2021

This comment has been minimized.

This comment has been minimized.

wooorm commented Jan 8, 2021

tvquizphd commented Jan 9, 2021 •

edited

Loading

wooorm commented Jan 9, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

tvquizphd commented Jan 10, 2021 •

edited

Loading

tvquizphd commented Jan 10, 2021 •

edited

Loading

This comment has been minimized.

tvquizphd commented Jan 15, 2021 •

edited

Loading

wooorm commented Jan 17, 2021

wooorm commented Jan 17, 2021

Fix caps, keyboard group logic #39

Fix caps, keyboard group logic #39

Conversation

tvquizphd commented Jan 3, 2021 • edited Loading

Addressing a bigger problem

It's ready to merge!

tvquizphd left a comment

Choose a reason for hiding this comment

tvquizphd Jan 3, 2021

Choose a reason for hiding this comment

tvquizphd Jan 3, 2021

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

wooorm commented Jan 8, 2021

tvquizphd commented Jan 9, 2021 • edited Loading

Practical results

Odd observation

wooorm commented Jan 9, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

tvquizphd commented Jan 10, 2021 • edited Loading

tvquizphd commented Jan 10, 2021 • edited Loading

This comment has been minimized.

tvquizphd commented Jan 15, 2021 • edited Loading

wooorm commented Jan 17, 2021

wooorm commented Jan 17, 2021

tvquizphd commented Jan 3, 2021 •

edited

Loading

tvquizphd commented Jan 9, 2021 •

edited

Loading

tvquizphd commented Jan 10, 2021 •

edited

Loading

tvquizphd commented Jan 10, 2021 •

edited

Loading

tvquizphd commented Jan 15, 2021 •

edited

Loading