Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Style navigation QuickNav command #16050

Merged
merged 33 commits into from
Apr 16, 2024
Merged

Style navigation QuickNav command #16050

merged 33 commits into from
Apr 16, 2024

Conversation

mltony
Copy link
Contributor

@mltony mltony commented Jan 16, 2024

Link to issue number:

Closes #16000

Summary of the issue:

Adding jump to same style and jump to different style QuickNav commands.

Description of user facing changes

Adding jump to same style and jump to different style QuickNav commands. They are not assigned to any keyboard gestures by default.

Description of development approach

Scanning document by paragraph in the desired direction and analyzing styles within each paragraph.

Testing strategy:

Tested manually in Chrome and Firefox.
Added a system test.

Known issues with pull request:

  • In addition to same style navigation I also added different style navigation command because it's implementation turned out to be very similar - I hope that's ok.
  • This PR will have conflicting changes with QuickNav Text paragraph navigation #16031 - I will resolve conflicts whenever either of these PRs is merged.
  • Another quirk to be aware of: in this PR I treat paragraphs separately. For example if we have an entire document written in the same style, then jump to next same style command would iterate over paragraphs. In theory we can try to merge paragraphs and parts of paragraphs written in the same style together, but this would make code more complicated and I personally feel current approach is more intuitive, but LMK if you think otherwise. Update: as requested, now I changed the code so that every chunk found by this pR can span multiple paragraphs.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@AppVeyorBot
Copy link

See test results for failed build of commit 3297106adb

@mltony
Copy link
Contributor Author

mltony commented Jan 16, 2024

The bot says some system test with tag Installer is failing. I couldn't find any information on waht test is failing. Not sure if this is flaky test. If it's not flaky - could someone help me to find which test is failing? It wasn't very clear to me based on bot's comment.

@mltony mltony marked this pull request as ready for review January 16, 2024 04:19
@mltony mltony requested review from a team as code owners January 16, 2024 04:19
@seanbudd
Copy link
Member

Yes I think that test failure is unrelated

source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
@seanbudd seanbudd added the conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review. label Jan 17, 2024
Copy link
Collaborator

@CyrilleB79 CyrilleB79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feature is really promising.

I have only a few small GUI / documentation changes to suggest.

I have not tested the code though.

source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
user_docs/en/changes.t2t Outdated Show resolved Hide resolved
@CyrilleB79
Copy link
Collaborator

In an Outlook message, I was hoping to jump to "answers in red" sent by a colleague of mine.

Unfortunately, it does not work. I get errors such as the following when calling all 4 commands (here with Move to next different style text command):

ERROR - scriptHandler.executeScript (16:39:49.269) - MainThread (9984):
error executing script: <bound method BrowseModeTreeInterceptor.addQuickNav.<locals>.<lambda> of <nvdaBuiltin.appModules.outlook.MailViewerTreeInterceptor object at 0x01DBFF90>> with gesture 'j'
Traceback (most recent call last):
  File "scriptHandler.py", line 295, in executeScript
    script(gesture)
  File "browseMode.py", line 508, in <lambda>
    script = lambda self,gesture: self._quickNavScript(gesture, itemType, "next", nextError, readUnit)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "browseMode.py", line 466, in _quickNavScript
    item = next(iterFactory(direction, info))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "browseMode.py", line 2111, in _iterTextStyle
    styles = self._extractStyles(initialTextInfo)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "browseMode.py", line 2075, in _extractStyles
    if field.field['role'] == controlTypes.Role.MARKED_CONTENT:
       ~~~~~~~~~~~^^^^^^^^
KeyError: 'role'

@CyrilleB79
Copy link
Collaborator

CyrilleB79 commented Jan 23, 2024

Regarding unassigned keys:
IMO, if this feature works as advertised, it would be much more interesting to have default assigned keys for it and keep QuickNav Text paragraph navigation commands unassigned (#16031).

Maybe we should merge this PR and #16031 with unassigned gestures and test them during alpha stage. Then at the end of alpha stage, we can figure if one or both of these feature need to have an assigned gesture and add them.

mltony and others added 4 commits January 25, 2024 12:30
Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>
Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>
Co-authored-by: Cyrille Bougot <cyrille.bougot2@laposte.net>
source/browseMode.py Outdated Show resolved Hide resolved
Co-authored-by: Łukasz Golonka <lukasz.golonka@mailbox.org>
@AppVeyorBot
Copy link

See test results for failed build of commit d32acb6519

source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
tests/system/robot/chromeTests.py Outdated Show resolved Hide resolved
@seanbudd seanbudd marked this pull request as draft January 29, 2024 06:13
mltony and others added 4 commits January 30, 2024 16:04
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
@CyrilleB79
Copy link
Collaborator

@mltony the feature does not seem to work with Word legacy; I did not re-test with UIA.

Could you double-check that all command types work in Word with and without UIA? Let me know when you have re-tested this successfully. Thanks.

@mltony
Copy link
Contributor Author

mltony commented Feb 15, 2024

@CyrilleB79 - as I mentioned #16171 and #16172 are word specific problems that prevent style navigation to work correctly in MSWord. Both issues affect both legacy and UIA word support. Until both are fixed, style navigation won't work correctly in MSWord.

@CyrilleB79
Copy link
Collaborator

@mltony, sorry to come a bit late with it, but I had not paid attention to the following known issue:

Another quirk to be aware of: in this PR I treat paragraphs separately. For example if we have an entire document written in the same style, then jump to next same style command would iterate over paragraphs. In theory we can try to merge paragraphs and parts of paragraphs written in the same style together, but this would make code more complicated and I personally feel current approach is more intuitive, but LMK if you think otherwise.wouldn't seem very logic

My personal opinion is that merging same style across paragraph boundaries is more than desirable. For me, the main use case of this PR is : "See my comments in red the message below". In a message with many bullet paragraphs (e.g. meeting minutes), with only few comments in red, I do not expect the jump commands to jump to each bullet.

@CyrilleB79
Copy link
Collaborator

@CyrilleB79 - as I mentioned #16171 and #16172 are word specific problems that prevent style navigation to work correctly in MSWord. Both issues affect both legacy and UIA word support. Until both are fixed, style navigation won't work correctly in MSWord.

Re #16171: as mentioned there, word is able to read text with formatting information if NVDA is configured for this. Thus you should probably use the same technique for style navigation.

Re #16172: I have not reviewed the code. But it seems strange to me that you cannot work it around, e.g. catching the exception to indicate the end of the document. Actually Word has a browse mode with a working quick nav capability. Why is style navigation impacted and other quick nav commands not?

I hope you can fix #16171 and #16172 in this PR. Especially because Outlook uses Word browse mode on by default, and because the "See my comments in red in the message below" is a very common use case.

Though, if you confirm that you cannot fix them, you should:

  • mention it in known issue section
  • disable these commands in Word/Outlook so that we do not expect them to work

@XLTechie
Copy link
Collaborator

XLTechie commented Feb 16, 2024 via email

@mltony mltony marked this pull request as draft February 16, 2024 18:50
@mltony
Copy link
Contributor Author

mltony commented Feb 18, 2024

@CyrilleB79, @XLTechie,
I addressed your comments. Now same style chunks can span across multiple paragraphs. This made the code significantly more complicated as I warned before. Took me quite some time to debug that.
Also I made this PR compatible with MS Word. I figured out how to retrieve formatting information there. Also I still think #16172 is a bug - as it shouldn't throw an error. But for now I am catching this error in my code, so this PR works as expected and is not blocked on #16172.

@mltony mltony marked this pull request as ready for review February 18, 2024 22:53
@CyrilleB79
Copy link
Collaborator

Thanks @mltony. This actually works much better in Word.

There still seems to be little issues with links of e-mails and URLs. Try to type the following:
My e-mail is foo.bar@example.com and I’ll provide a link on https://github.com/nvaccess/nvda/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc. Is it OK.

Depending on where you have initially put the cursor, you get unexpected results. E.g. cursor before the link, or after its first chararacter, or in the middle, or on the penultimate character, or on the ultimate character, or just after the last character of the link, etc.
I wonder if it's related to punctuation symbols or to some word strange behaviour.

@mltony
Copy link
Contributor Author

mltony commented Feb 20, 2024

@CyrilleB79 ,
I debugged this and drilled it down to #16196 which happnes to be a duplicate of #11427, which according to Sean is blocked on an external fix.
It seems that the issue is only applicable to links in MSWord. So I worked around this by ignoring color of links in MSWord.

@mltony mltony marked this pull request as draft February 21, 2024 18:28
michaelDCurran pushed a commit that referenced this pull request Apr 8, 2024
This function is needed for both #8518 and #16050.

Summary of the issue:
Suppose we have TextInfo that represents a paragraph of text:
```
> s = paragraphInfo.text
> s
'Hello, world!\r'
```

Suppose that we would like to put the cursor at the first letter of the word 'world'.
That means jumping to index 7:
```
> s[7:]
'world!\r'
```

The problem is that calling paragraphInfo.move(UNIT_CHARACTER, 7, "start") is not guaranteed to achieve desired effect. There are two main reasons for that:
1. In Wide character encoding, some 4-byte unicode characters are represented as two surrogate characters, whereas in pythonic string they would be represented by a single character.
2. In non-offset TextInfos (e.g. UIATextInfo) there is no guarantee on the fact that TextInfos.move(UNIT_CHARACTER, 1)would actually move by exactly 1 character. A good illustration of this is in Microsoft Word with UIA enabled always, the first character of a bullet list item would be represented by three pythonic characters: 
◦ Bullet character "•"
◦ Tab character \t
◦ And the first character of of list item per se.
3. The third problem of TextInfo.move(UNIT_CHARACTER) function is its performance in some implementations. In particular, moving by 10000 characters in Notepad++ takes over a second on a reasonably modern PC. I might not need 

to move by 10000 characters in my upcoming PRs, but I will need to move by a few thousands for sure since for sentence navigation I would need to move within a paragraph and some large paragraphs in typical texts can easily be few thousands characters. I need to find both beginning and end textInfos, and if each operation takes say 200ms, then we'd be wasting almost half a second on just moving by characters. Since there were previous concerns about sentence navigation being not fast enough, II would like to introduce this efficient implementation.
Here is how this can be done efficiently using this PR:

```
> info = paragraphInfo.moveToPythonicOffset(7)
> info.setEndPoint(paragraphInfo, "endToEnd")
> info.text
'world!\r'
```

Description of development approach
1. For general case, I implemented binary-search-like algorithm. I explained it in great detail in the code. Please see def moveToPythonicOffset function in textInfos\__init__.py.
2. I provided optimized implementations for OffsetsTextInfo and CompoundTextInfo.
3. I refactored textUtils.py making it conformant to OOP style. I implemented UTF8OffsetConverter and dummy IdentityOffsetConverter as well as their abstract base class and a function getOffsetConverter that selects correct converter based on encoding. I renamed a couple of methods of WideStringOffsetConverter in order to remove the word wide - as now I would like to use similar methods for UTF8 converter, and it has nothing to do with wide strings.
@mltony mltony marked this pull request as ready for review April 9, 2024 18:16
@mltony
Copy link
Contributor Author

mltony commented Apr 9, 2024

I updated this to use moveToCodepointOffset function - this is now ready for review. I also fixed a small bug in moveToCodepointOffsets and added more unit tests to cover that bug.

Copy link
Member

@seanbudd seanbudd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes generally look good to me

source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
source/browseMode.py Outdated Show resolved Hide resolved
tests/unit/test_textInfos.py Outdated Show resolved Hide resolved
Copy link
Member

@Qchristensen Qchristensen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple change for the user guide etc, reads well.

mltony and others added 2 commits April 15, 2024 13:41
Co-authored-by: Sean Budd <seanbudd123@gmail.com>
Copy link
Member

@seanbudd seanbudd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mltony

@seanbudd seanbudd merged commit 8af4bf4 into nvaccess:master Apr 16, 2024
1 check passed
@nvaccessAuto nvaccessAuto added this to the 2024.2 milestone Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: add QuickNav command for Same style navigation in browse mode
9 participants