Updates to CID-keyed UFO handling #1628

punchcutter · 2023-03-23T11:04:53Z

Added getGlyphByCID to uforead
@kaydeearts this is on top of your kd-fix-glyphname-search branch. We need the same CID functionality as in t1read and cffread so this is one of those things. We should also check the -decid flag and make sure we can do that with UFO as well.

punchcutter · 2023-03-23T11:36:57Z

For reference we previously had an error fatal(h, "Cannot read glyphs from UFO fonts by CID "); when running a command like tx -g /12345 -ufo -o subset.ufo full.ufo
With this PR we can read by CID.

punchcutter · 2023-03-23T13:11:41Z

I didn't run tests or add any new yet. Just want to get this out there first so we can look at it.

punchcutter · 2023-03-25T11:37:14Z

Related to the absfont_dump changes I made I noticed that dumping a CID-keyed UFO doesn't report CIDCount because it's not set in uforead until after we have read the glyphs in ufoIterateGlyphs. Other formats already contain cid.CIDCount at the top. Perhaps we should also write CIDCount into the lib.plist along with ROS and CIDFontName. Then we can read it back without going through every glyph first. That would also make it faster to get the value when reading the UFO with Python.

Update: we should be setting it in parseCIDMap instead of ufoIterateGlyphs. Then it works as expected. I pushed that commit which makes more sense since reading the cidmap is where that info comes from in the first place.

punchcutter · 2023-03-25T11:51:12Z

With the last commit we get output like this from a CID-keyed UFO that also has real glyph names. By default we write UFOs with glyph names like cidXXXXX, but once in UFO they can be anything so we want to display them here. This also means we can convert to name-keyed and maintain the nice names.

## glyph[tag] {name,cid,iFD,LanguageGroup}
glyph[0] {.notdef,0,5,1}
glyph[1] {uni0020,1,14,0}
glyph[2] {uni0021,2,14,0}
glyph[3] {uni0022,3,14,0}
glyph[4] {uni0023,4,14,0}
glyph[5] {uni0024,5,14,0}
glyph[6] {uni0025,6,14,0}
glyph[7] {uni0026,7,14,0}
glyph[8] {uni0027,8,14,0}
glyph[9] {uni0028,9,14,0}
glyph[10] {uni0029,10,14,0}
glyph[11] {uni002A,11,14,0}
glyph[12] {uni002B,12,14,0}
glyph[13] {uni002C,13,14,0}
glyph[14] {uni002D,14,14,0}
glyph[15] {uni002E,15,14,0}
glyph[16] {uni002F,16,14,0}
glyph[17] {uni0030,17,14,0}
glyph[18] {uni0031,18,14,0}

punchcutter · 2023-03-26T06:33:43Z

I keep forgetting to put [skip ci] in these latest commits when I know tests will fail because of something unrelated.

…s [skip ci]

punchcutter · 2023-03-28T12:23:54Z

I made a few changes to writing the lib.plist. The CIDMap entry keys need to be ordered alphabetically or we fail in the uforead bsearch when checking glyphs. But we also want the public.glyphOrder to be sorted by CID. So now we get both. With this I can round trip dumping 18 FDicts as subset CID-keyed UFOs and then running mergefonts to put them back together. The only thing missing for that is that mergefonts doesn't yet understand how to maintain glyph names when merging UFOs, but that's probably not a big deal right now.

…warning because the dup glif is discarded earlier when calling findGLIFRecByName(h, glyphName). Previously, this was searching by fileName rather than glyphName.

…vious memory. More work needed for this in the future.

…e have different inputs and different parsing functions that aren't all consistent yet. Better fixes in the future.

skef · 2023-04-04T09:59:52Z

c/shared/source/absfont/absfont_dump.c

-        FPRINTF_S(h->fp, "## glyph[tag] {cid,iFD");
-    else
+        /* UFO can store names even when CID-keyed */
+        if (top->sup.srcFontType == 7) {


Is there, or could there be, some constant defined to use instead of 7 here?

Duh, I have no idea why I put 7. The enum is abfSrcFontTypeUFOCID

Updated with the enum string instead of 7.

skef · 2023-04-04T10:02:38Z

c/shared/source/tx_shared/tx_shared.c

-    else
-        sprintf(gname, "cid%hu", info->cid);
+    if (info->gname.ptr != NULL) {
+        strcpy(gname, info->gname.ptr);


Using strncpy seems particularly wise here given the fixed buffer size and the potentially unverified length.

skef · 2023-04-04T10:13:37Z

c/shared/source/ufowrite/ufowrite.c

@@ -340,6 +340,86 @@ int ufwBegFont(ufwCtx h, long flags, char *glyphLayerDir) {
    return ufwSuccess;
 }

+static void orderNameKeyedGlyphs(ufwCtx h) {


I guess we're using these n^2 sorts for stability?

I don't think all this sorting is lovely, but cffwrite and t1write have ordering by CID or Name so I half copied what's done in those so that we can correctly sort here. The reason it came up was because we were writing the CIDMap in CID order which then made uforead fail to read it correctly, but more importantly if I open and save the UFO in typical UFO Python tools then the lib.plist was in a completely different. The whole XML dict is sorted by key including CIDMap and we weren't doing that in ufowrite so that's what all this does. Order by name so that we can write the CIDMap correctly and then order by CID so that public.glyphOrder is in the correct CID order. It's kind of ugly. We should really be reading all of this into a libxml2 structure and work from that, but this all works well enough so far.

I was thinking more about the efficiency than presence of the sorting, but I guess the worst case is 2^32 operations, which isn't a deal breaker.

I believe we use a canned quicksort elsewhere, which is why I asked about stability (quicksort isn't stable), but it seems like it's doubtful we'd have duplicate keys anyway (in which case stability isn't relevant).

I dunno, I guess we can do this for the time being and revisit later.

skef

lgtm

kaydeearts

Skef had already approved & I'm adding my 👍 !

kaydeearts and others added 2 commits March 22, 2023 16:17

[uforead] use glyphName instead of fileName for glyph search

86a8991

Let tx read UFO glyphs by CID like in t1read and cffread.

416f508

punchcutter requested a review from kaydeearts March 23, 2023 11:04

Allow glyph names in CID-keyed UFOs

3aa6175

Dump glyph name for CID-keyed UFOs

84ce922

Update tx dump of CID-keyed UFOs with glyph names

4b4343e

punchcutter mentioned this pull request Mar 25, 2023

[tx] decid and mergefonts do not correctly handle glyph names in CID-keyed UFOs #1631

Closed

punchcutter added 2 commits March 26, 2023 13:26

Move setting of CIDCount in uforead to parseCIDMap

b461971

Add orderCIDKeyedGlyphs to keep CID order in UFOs

8fca2ae

punchcutter changed the title ~~Add getGlyphByCID to uforead~~ Updateds to CID-keyed UFO handling Mar 26, 2023

punchcutter changed the title ~~Updateds to CID-keyed UFO handling~~ Updates to CID-keyed UFO handling Mar 26, 2023

punchcutter added 3 commits March 26, 2023 19:40

Remove unused dicts in FDArray

d4ee093

Let CID-keyed UFO -> Type 1 name-keyed conversion maintain glyph name…

3d5cdd9

…s [skip ci]

Update CID-keyed UFO glyph sorting

be24ebf

kaydeearts added 2 commits March 29, 2023 14:24

[tx_test] A duplicate glif with the same name no longer creates this …

ba2e048

…warning because the dup glif is discarded earlier when calling findGLIFRecByName(h, glyphName). Previously, this was searching by fileName rather than glyphName.

[absfont_dump] Ubuntu flake8 fixes

a526a83

kaydeearts force-pushed the zqs-fix-ufo-GlyphByCID branch from 5c064ea to a526a83 Compare March 29, 2023 21:24

kaydeearts added 2 commits March 31, 2023 15:12

[tx_test] Add test case with running decid on cid UFO with FDArray

c5287a1

[tx_shared] Properly re-alloc memory for selected FD and free the pre…

17e6879

…vious memory. More work needed for this in the future.

kaydeearts force-pushed the zqs-fix-ufo-GlyphByCID branch from ef164a5 to 17e6879 Compare March 31, 2023 22:12

[tx_shared.c] Should not assume memory is allocated for this ptr as w…

9c9389a

…e have different inputs and different parsing functions that aren't all consistent yet. Better fixes in the future.

skef reviewed Apr 4, 2023

View reviewed changes

Address code review comments from Skef

0d217b6

kaydeearts requested a review from skef April 5, 2023 20:51

[uforead] WARN instead of FAIL when glyph is missing CID number

8cd1e6d

skef previously approved these changes Apr 5, 2023

View reviewed changes

[tx_data] Fix some tests for fail deprecated to warn

372b1c3

kaydeearts dismissed skef’s stale review via 372b1c3 April 5, 2023 22:47

kaydeearts requested a review from skef April 5, 2023 22:47

kaydeearts added 5 commits April 5, 2023 15:55

[tx_data] Add needed test files

ba62fdd

[tx_test] Should use pfb, not pfa, in expected_outputs

ff5ba8c

[uforead] Check if CID is defined before adding char

aea404a

[tx_data] fix expected outputs + input for tests, + cpplint fix

b03357f

[uforead, tx_test] Clarify warning regarding missing CID number

0d7a5ea

kaydeearts force-pushed the zqs-fix-ufo-GlyphByCID branch from 42a9390 to 0d7a5ea Compare April 6, 2023 04:17

kaydeearts approved these changes Apr 6, 2023

View reviewed changes

kaydeearts merged commit 23ddd12 into develop Apr 6, 2023
7 checks passed

kaydeearts deleted the zqs-fix-ufo-GlyphByCID branch April 6, 2023 04:27

kaydeearts mentioned this pull request Apr 6, 2023

[tx] revert missing cid fail to warn #1633

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates to CID-keyed UFO handling #1628

Updates to CID-keyed UFO handling #1628

punchcutter commented Mar 23, 2023 •

edited

punchcutter commented Mar 23, 2023

punchcutter commented Mar 23, 2023

punchcutter commented Mar 25, 2023 •

edited

punchcutter commented Mar 25, 2023

punchcutter commented Mar 26, 2023

punchcutter commented Mar 28, 2023

skef Apr 4, 2023

punchcutter Apr 4, 2023

punchcutter Apr 5, 2023

skef Apr 4, 2023

punchcutter Apr 5, 2023

skef Apr 4, 2023

punchcutter Apr 4, 2023

skef Apr 4, 2023

skef left a comment

kaydeearts left a comment

Updates to CID-keyed UFO handling #1628

Updates to CID-keyed UFO handling #1628

Conversation

punchcutter commented Mar 23, 2023 • edited

punchcutter commented Mar 23, 2023

punchcutter commented Mar 23, 2023

punchcutter commented Mar 25, 2023 • edited

punchcutter commented Mar 25, 2023

punchcutter commented Mar 26, 2023

punchcutter commented Mar 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skef left a comment

Choose a reason for hiding this comment

kaydeearts left a comment

Choose a reason for hiding this comment

punchcutter commented Mar 23, 2023 •

edited

punchcutter commented Mar 25, 2023 •

edited