Read HTML entities, unicode characters, other symbols #3805

Open
nvaccessAuto opened this Issue Jan 22, 2014 · 34 comments

2 participants

@nvaccessAuto

Reported by paulbohman on 2014-01-22 23:33
Many HTML entities, unicode characters, and other symbols are not read by default, or at all. When web developers or content writers put these characters or symbols in their content, it's almost always because they're using them to convey some meaning. There are exceptions, of course, when symbols may be used for decorative purposes, but I don't think that's the norm.

As an example, NVDA reads the left and right arrow HTML entities (← and →), but for some reason NVDA doesn't read up arrow or down arrow. When web authors use these characters, it's usually because they are conveying some meaning, like up to the next level, or down a level, or next page or previous page. Or maybe they're using them to explain the NVDA shortcut keys: Control plus alt plus up arrow, for example.

Similarly, symbols like the dagger or double dagger symbol might be used for footnotes. There are a lot of other characters and symbols out there -- and I realize that the magnitude of the lists of characters is an issue -- but in most cases they're used to convey meaning, so I would want them read by default.

For most of them, it's enough to simply say what the character is: "dagger" or "heart" or whatever the symbol is. I wouldn't worry about trying to interpret "I heart you" and changing it to "I love you." Just say what the symbol is.
Blocking #3752

@nvaccessAuto

Comment 1 by paulbohman on 2014-01-22 23:58
I realize I may need to be more specific and come up with a list of characters that I would recommend having read out loud, because I obviously don't want the screen reader to say things like "carriage return" or "horizontal tab," which are unicode characters. Let me offer these as a starting point for the conversation, using the list of unicode sets at http://en.wikipedia.org/wiki/List_of_Unicode_characters:

Latin 1-Supplement:

  • characters U+00A2 through U+00BE

Arrows:

  • all, from U+2190 to U+21Fx

Mathematical operators:

  • all, from U+2200 to U+22Fx

Geometric shapes:

  • all, from U+25A0 to U+25FF

Miscellaneous symbols:

  • all, from U+2600 to U+26FF

Dingbats

  • all, from U+2701 to U+27BF

Supplemental arrows-A:

  • all, from U+27F0 to U+27FF

Miscellaneous symbols and arrows:

  • all, from U+2B00 to U+2B59

Emoji:

  • all
@nvaccessAuto

Comment 2 by jteh on 2014-01-23 00:27
This is fine, but for two things:
1. We do need to be careful about how many symbols we define, as the more there are, the longer the symbol data will take to load. This will need to be judged on a case by case basis.
2. The names need to be as short as possible, so it's not simply a matter of importing the Unicode names. This does need human intervention. :)

This is likely to be a pretty time consuming job, but it is just a matter of editing a file with tab separated values and following the documentation, so marking as goodForNewDev.

@nvaccessAuto

Comment 3 by paulbohman on 2014-01-23 00:35
Thank you!

And believe me, I know how time consuming it is. Just writing the blog entry that I wrote took many hours of typing and testing.

As far as HTML entities go, I'm not sure where the most authoritative source is. This one seems pretty inclusive: http://dev.w3.org/html5/html-author/charref

Of these, my personal highest priorities would be anything that has to do with math, plus arrows, plus things like section, paragraph, and superscripts. There may be a few others that I'd like, but those come to mind.

@nvaccessAuto

Comment 4 by paulbohman on 2014-01-23 00:45
By the way, if you need to combine this issue with the last one that I posted, feel free to do that. I meant for the two to refer to slightly different things, with one talking about keyboard punctuation and another talking about unicode and HTML entities, but I didn't do a good job of distinguishing them in my descriptions. Sorry about that.

@nvaccessAuto

Comment 5 by driemer.riemer@... on 2014-01-23 02:58
How do I do this? I am interested in working on the code.
Would it be in the punctuation data in the locale folder in symbols.dic? If so won't this data need translated?

@nvaccessAuto

Comment 6 by briang1 on 2014-01-23 08:32
I am not sure about the shortness rule. I found when i wrote my version of the existing file to make _ into underscore and ! into Exclamation that although longer, they were easier to interpreet to me at least.

Also, what happens, for example in Usenet newsgroups where I think greater than or less than is used to denote quoted text. I have made sure these are only spoken in all, but it might well be that one needs to say, quoted text the first time one encounters the symbol in a news client.

Also presumably, these will need manually editing for each instance for all the languages in nvda by the translators. There are a heck of a lot.

@nvaccessAuto

Comment 7 by nvdakor on 2014-01-23 10:15
Hi,
Yes, you'll need to modify source/locale/en/symbols.dic, and we the translators will get a diff folder with new symbols added. However, as Jamie pointed out, we should be careful about which symbols we should add.
For superscripts and subscripts, we could work around this via regexp speech rule so that NVDA will speak superscript number (if these are located next to each other in Unicode tables).
Good luck.

@nvaccessAuto

Comment 8 by nishimotz on 2014-01-23 12:32
Regarding this, Unicode 6.0 Emoji characters, as well as some CJK ideographic characters, are out of Unicode Basic Multilingual Plane, and they can not be handled correctly even if they are in the symbol or character description tables.
According to #2090, work around the surrogates are very limited so far.
Is this correct?

@nvaccessAuto

Comment 9 by driemer.riemer@... on 2014-01-23 22:38
so if I work on this, what symbols should be prioritized? Also, how do I get my changes to
nvaccess?

@nvaccessAuto

Comment 10 by jteh (in reply to comment 8) on 2014-01-23 23:00
Replying to nishimotz:

Regarding this, Unicode 6.0 Emoji characters, as well as some CJK ideographic characters, are out of Unicode Basic Multilingual Plane, and they can not be handled correctly even if they are in the symbol or character description tables.

As I understand it, the symbol table should be able to handle this; it can handle multiple codepoints in the same entry. The issue is more whether APIs can handle passing multi-codepoint characters to us. Some work might be needed on speak spelling/character descriptions.

@nvaccessAuto

Comment 11 by jteh (in reply to comment 9) on 2014-01-23 23:01
Replying to driemer.riemer@…:

how do I get my changes to

nvaccess?

Ideally, a Git branch. Failing that, a patch. As a last resort, just the modified file, but please tell us which revision of the code it is based on. Always base changes on the master branch. Thanks.

@nvaccessAuto

Comment 12 by JohnHoltRipley on 2014-02-26 13:54
If it helps at all, I've been researching how other Screen Readers handle Unicode characters, and the results are here:
http://unicode.johnholtripley.co.uk/all/screenreader

@nvaccessAuto

Comment 13 by nvdakor on 2014-03-13 23:18
Hi,
I do know that there is an add-on which reads Emoticons.
For Derek: I'll go ahead and create an experimental t3805 branch on the community dev repo.

@nvaccessAuto

Comment 14 by nvdakor on 2014-03-13 23:22
Hi all,
For devs interested in working on this branch, I have created a community development repo at:

@nvaccessAuto

Comment 15 by driemer.riemer@... on 2014-08-17 04:52
So I think that some math homework systems show their symbols with the raw unicode. I am going to add this support to my git fork and then commit this. I will try throwing in some other symbols when there.
https://derek_riemer_@bitbucket.org/derek_riemer_/nvda.git
I will create this in t3805 and see what happens here as I git going on this.

@nvaccessAuto

Comment 18 by ajirving on 2014-10-23 18:17
Adding more symbols is a good idea if they're used sufficiently often, but we presumably don't want to have the entire unicode table. Why not add a function which uses the python unicodedata module to lookup the name of a character? It could be bound to the read character key pressed 4 times.

@nvaccessAuto

Comment 19 by jteh (in reply to comment 18) on 2014-10-23 21:49
Replying to ajirving:

Why not add a function which uses the python unicodedata module to lookup the name of a character?

The problem with this is that it is English only and cannot be localised.

@nvaccessAuto

Comment 20 by siddhartha_iitd on 2014-12-01 07:08
I've added some characters and their respective descriptions in symbols.dic file. Following categories of characters are included:
Mathematical Operators
Arrows
Dingbats
Geometric Shapes
Miscellaneous Symbols
Emoticons
Superscripts & Subscripts

For some characters, no standard description could be found. So, such characters are included without any description. If we want to retain such characters, we might have to go with non-standard descriptions.

Please check the branch in_t3805 by following the below mentioned url:
https://manish_agrawal@bitbucket.org/manish_agrawal/nvda.git

@nvaccessAuto

Comment 21 by jteh on 2014-12-01 10:04
Please remove any characters without descriptions, as these aren't useful to users. Perhaps someone will subsequently report the need for one of these characters, but in that case, they will also hopefully be able to provide a useful description. Thanks.

@nvaccessAuto

Comment 22 by siddhartha_iitd (in reply to comment 21) on 2014-12-01 11:24
Replying to jteh:

Please remove any characters without descriptions, as these aren't useful to users. Perhaps someone will subsequently report the need for one of these characters, but in that case, they will also hopefully be able to provide a useful description. Thanks.

Thanks for quick reply! The characters without any standard description are removed from symbols.dic file. The updated symbols.dic file is available in branch '''in_t3805''' at following url:
https://manish_agrawal@bitbucket.org/manish_agrawal/nvda.git

@nvaccessAuto

Comment 24 by driemer.riemer@... on 2014-12-01 15:05
If people could make a list of the symbols they would find useful, so we can find the common symbols, that would be great. It also might be good to block this with the ticket to let users add their own custom symbols as well.

@nvaccessAuto

Comment 25 by jteh (in reply to comment 24) on 2014-12-01 21:56
Replying to driemer.riemer@…:

If people could make a list of the symbols they would find useful, so we can find the common symbols, that would be great.

I suggest we start with the changes proposed in comment:20. If you're interested, it'd be good if you can take a look at that.

It also might be good to block this with the ticket to let users add their own custom symbols as well.

Blocking doesn't make sense. This is about the default experience. Customising is a complementary feature, but not a requirement for improving the default experience.

@nvaccessAuto

Comment 26 by dineshkaushal on 2014-12-06 07:14
How about using unicodedata.name instead of defining all these symbols?

@nvaccessAuto

Comment 27 by jteh (in reply to comment 26) on 2014-12-06 10:19
Replying to dineshkaushal:

How about using unicodedata.name instead of defining all these symbols?

The Unicode names are English only and overly verbose for screen reader use.

@nvaccessAuto

Comment 28 by dineshkaushal on 2015-03-26 11:17
I was reading some mathematical Equations, and I found NVDA was not reading some symbols such as summation sign and sign for beta. I tried branch in_t3805 which is created by Siddhartha Gupta, and I could read those symbols.

Is this branch going to be part of 15.2?

@nvaccessAuto

Comment 29 by siddhartha_iitd on 2015-03-26 14:17
Added small and capital Greek Alphabets. The updated file is available in branch in_t3805 at following url:
https://bitbucket.org/manish_agrawal/nvda.git

@nvaccessAuto

Comment 30 by jteh on 2015-04-14 07:11
We're going to do this in phases:
1. Take the math symbols as is. Mick will do this.
2. Investigate how to handle Greek letters without breaking Greek. I'll handle this.
3. Investigate other common symbols for inclusion. This one is controversial, but the reality is that I don't want to include a massive number of symbols that will rarely be used for performance and translation reasons. The names also need to be as brief as possible (some of the names in the proposed patch are overly descriptive for a screen reader) and hand editing all of these will be painful.
4. Investigate a way to use the English Unicode names as a last resort, so that even if all else fails, the user can get something. I imagine we'd do this as a character description rather than trying to report these normally. Aside from avoiding verbosity, this is also necessary because we don't want characters that would normally be pronounced by the user's synthesiser being clobbered by this. (It's important to note that we have no idea what characters various synths will report and synths don't provide any indication of this.)

@nvaccessAuto

Comment 31 by Michael Curran <mick@... on 2015-04-30 04:57
In [3f73d94]:
```CommitTicketReference repository="" revision="3f73d94efeff8fe1c986e6ff8ed7ebcf9cff2fa7"
Merge branch 't3805' into next. Incubates #3805

Changes:
Added labels: incubating
@nvaccessAuto

Comment 32 by Michael Curran <mick@... on 2015-06-19 19:11
In [b5d0db7]:
```CommitTicketReference repository="" revision="b5d0db74abbbdf01d4f601bcce6674030e675b08"
Merge branch 't3805'. Fixes #3805

Changes:
Removed labels: incubating
State: closed
@nvaccessAuto

Comment 33 by mdcurran on 2015-06-19 19:15
Changes:
Milestone changed from None to 2015.3

@nvaccessAuto

Comment 34 by mdcurran on 2015-06-19 21:27
Reopening as only math symbols were provided in that fix.
Changes:
State: reopened

@nvaccessAuto

Comment 35 by jteh on 2015-06-26 06:10
Moving out of 2015.3 for the remaining work.
Changes:
Milestone changed from 2015.3 to None

@nvaccessAuto

Comment 38 by JamaicanUser on 2015-07-12 16:12
I notice that the following symbols were not added to the Master snapshot.
1. the heart/spade/diamond/club suits. They can be used as bullets.
2. the up/down/horizontal/left double/right double/up double/down double/horizontal double arrows.
3. the left/right angle brackets.
4. prime/double prime.
5. the middle dot (small bullet)
6. not/almost equal to sign
7. dagger/double dagger
8. paragraph mark
9. Dutch Florin
10. broken/horizontal bar.

@raebened

Request addition of the Unicode star symbols. 2726, 2605, 2736, 2734, 2739, and 2735. These are used as bullets and in tables as indicators like the checkmark symbol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment