Move char codes from high surrogate pair range into private use. #9062
Conversation
|
Looks like I need to fix some unit tests, but in the mean time... /botio test |
From: Bot.io (Windows)ReceivedCommand cmd_test from @brendandahl received. Current queue size: 0 Live output at: http://54.215.176.217:8877/ad1fb46a42805ed/output.txt |
From: Bot.io (Linux m4)ReceivedCommand cmd_test from @brendandahl received. Current queue size: 0 Live output at: http://54.67.70.0:8877/7f7d10abb9ece3c/output.txt |
From: Bot.io (Linux m4)FailedFull output at http://54.67.70.0:8877/7f7d10abb9ece3c/output.txt Total script time: 16.88 mins
|
From: Bot.io (Windows)FailedFull output at http://54.215.176.217:8877/ad1fb46a42805ed/output.txt Total script time: 22.92 mins
|
@brendandahl Do you want to fix the test failures, such that we can land this PR as a temporary solution? |
|
@Snuffleupagus I'll fix failures as a temporary solution. I started working on the full solution, but when I start moving more glyphs to the private use area it causes more problems. If a glyph has an id of 0 (usually notdef, but in some cases a valid glyph) and it gets moved the private use area it no longer will be drawn by the browser. What we really need to do is shift all the glyphs and insert a fake notdef, but that's a non-trivial amount of work. |
|
@brendandahl Ping :-) |
| @@ -83,7 +83,7 @@ var checkProblematicCharRanges = function checkProblematicCharRanges() { | |||
|
|
|||
| describe('Fonts', function() { | |||
| it('checkProblematicCharRanges', function() { | |||
| var EXPECTED_PERCENTAGE = 45; | |||
| var EXPECTED_PERCENTAGE = 100; | |||
There was a problem hiding this comment.
Is it strictly necessary to make this a full 100 percent, for the tests to pass with this patch?
The reason that I'm asking is that with this change, there's nothing stopping us from adding enough entries to ProblematicCharRanges that there'd be no room over for re-mapping glyphs that would otherwise end up at a previously mapped location. Basically, wouldn't we always hit
Lines 787 to 790 in 518da6c
100 this test-case would ensure that there's atleast some space left in the PUA?
There was a problem hiding this comment.
I think it's now at 70%. I could lower it, but it seems odd to have a test that we have to change every time the ranges change. We would only run out of space in the PUA if the font had tons of values without unicode values and lots of values with problematic unicodes (or a very large symbolic font).
There was a problem hiding this comment.
Sure, let's just do this for now!
Hopefully, we'd be able to (eventually) replace the current black-list approach with a white-list instead.
|
/botio test |
From: Bot.io (Windows)ReceivedCommand cmd_test from @Snuffleupagus received. Current queue size: 0 Live output at: http://54.215.176.217:8877/9e08853f71b5e53/output.txt |
From: Bot.io (Linux m4)ReceivedCommand cmd_test from @Snuffleupagus received. Current queue size: 0 Live output at: http://54.67.70.0:8877/8a4ec69060f5efc/output.txt |
From: Bot.io (Linux m4)SuccessFull output at http://54.67.70.0:8877/8a4ec69060f5efc/output.txt Total script time: 17.00 mins
|
From: Bot.io (Windows)SuccessFull output at http://54.215.176.217:8877/9e08853f71b5e53/output.txt Total script time: 23.30 mins
|
|
/botio makeref |
From: Bot.io (Linux m4)ReceivedCommand cmd_makeref from @Snuffleupagus received. Current queue size: 0 Live output at: http://54.67.70.0:8877/6dcbfc3f0bf3ea6/output.txt |
From: Bot.io (Windows)ReceivedCommand cmd_makeref from @Snuffleupagus received. Current queue size: 0 Live output at: http://54.215.176.217:8877/c11218fefcc69f2/output.txt |
From: Bot.io (Linux m4)SuccessFull output at http://54.67.70.0:8877/6dcbfc3f0bf3ea6/output.txt Total script time: 15.74 mins
|
From: Bot.io (Windows)SuccessFull output at http://54.215.176.217:8877/c11218fefcc69f2/output.txt Total script time: 21.06 mins
|
Move char codes from high surrogate pair range into private use.
Fixes #2884
In the test PDF, the glyph that draws '4' has a unicode value of 0xD860 (which is not a valid unicode value). This value seems to have issues with Firefox and Chrome rendering it in that position.
I was hoping to stop playing whack-a-mole with the problematic unicode values and instead come up with a white list, but this is going to need more investigation. Stepping through Firefox's fillText canvas command I couldn't find an obvious spot where Firefox decided if it will draw a glyph at a certain char code. I'm thinking it's partially Firefox and the various font backends, but I'm not sure.
As another way to try and come up with a white list, I wrote a test script that uses a font that maps all char codes to one glyph and then attempted to draw all char codes from 0 to 0xFFFF. Unfortunately, Firefox and Chrome have different behavior. Firefox seems to have issues with lots of ranges, while Chrome doesn't have any issues. For Chrome, what's really strange is 0xD860 seems to work fine with the test script, but it also fails to render the attached PDF with pdf.js, so what Chrome draws may be dependent on font type.