Relationship between stem blending and ordering open to question

[skef]

CharString stems are specified with one of the four operators hstem, hstemhm, vstem, and vstemhm.

Section 4.3 states "It is required that the stems are encoded in ascending order (defined by increasing bottom (left) edge)." about hstem and (vstem) respectively. The stem order is 1:1 with the bits in a hintmask or cntrmask operand.

Section 3.1 states "Any operand for the hint and path construction operators may be replaced by the blend operator and its operands." Blending was added to support variable fonts as part of CFF2, while the definitions of the hint stem operators were left largely unaltered.

The current specification does not clarify the relationship between stem ordering requirement and the blend operator. There are two likely candidates:

1. Stems must stay in the same order in all points of design space.
2. Stems must be ordered according to their default values but can vary in order at other locations in design space.

The first candidate is the "safest", in that a font designed this way will likely work on any CFF2 rasterizer that took a CFF rasterizer as its starting point. However, the second is better for the standard. The stem of a design element that should be hinted can move "across" the stems of other elements because the element itself moves. The need for support of out-of-order stems at some points in design space is analogous to the need for overlapping shapes in variable glyphs: what is just be a matter of normalization in a static font becomes unnecessarily restrictive in a variable font.

Therefore the specification should specify the second candidate. The question is whether it can simply be updated to read that way or whether adding that language should be considered a revision, perhaps even requiring a version change (although what practical effect that version would have on older implementations is unclear).

Note that the consequence of rendering a glyph with stem order changes on a system that doesn't expect them is likely to be modest. Only a small number of glyphs in any given font are likely to have any stem ordering changes. The main advantage of ordered stems in a rasterizer is that the list can be binary-searched. Unaddressed ordering changes will cause some stems to be missed, with a rasterized result somewhere between a fully hinted version and an unhinted version.

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

• ID: d36330ec-abae-705d-5953-83f246aa3536
• Version Independent ID: bbb4020e-f2a2-93a3-c02a-187ae372db28
• Content: CFF2 CharString Format (OpenType 1.9) - Typography
• Content Source: typographydocs/opentype/spec/cff2charstr.md
• Service: typography
• GitHub Login: @PeterCon
• Microsoft Alias: alib

[Lorp]

As well as allowing binary search, another motivation for the encoded ordering may have been that, because of the delta encoding, the numbers will be smaller and therefore possibly need fewer bytes.

[PeterConstable]

It appears the same issue would apply to StemSnapH, StemSnapV and OtherBlues.

[skef]

@PeterConstable I'm not sure if you mean the whole issue or @Lorp's observation, but in case the former ...

The BlueValues question is somewhat different because:

1. There tend to be significantly fewer alignment zones than stems
2. Not only can alignment zones not overlap, but they must be a certain distance apart. (There is no equivalent of the hintmask mechanism.)
3. CFF2 allows multiple PrivateDICTs, which were previously only allowed in a CID context.

The way alignment zones are used in practice in Latin fonts is that there is typically only two alignment zones per glyph: the baseline and the top. One might also want ascender and descender zones for lower case (although those have been omitted in the from the whole grouping as they were seen as when everything was bundled together in one DICT they caused more trouble than they were worth). So, for example, if you have a dedicated axis that adjusts the height of smallcaps from between x-height and cap-height, you just need to ensure they have their own dictionary. And at Adobe we're now building our CFF2 latin-greek-cyrillic fonts with separate dictionaries for character types that typically only have two zones per dict.

Alignment zones are used somewhat more freely in CJK fonts, but as of now no one has raised a concern about a need for crossing alignment zones, and it's not as straightforward to add a mechanism for them to overlap, let alone cross each other. So Adobe is not currently concerned about the limitation.

As for StemSnapH and StemSnapV, those operators only encode widths. I believe that if you take any list of sorted numbers and linearly interpolate it with any other list of sorted numbers of the same length, or multiple such other lists arranged in a space as in a variable font, the result will always be sorted.

[PeterConstable]

I wasn't thinking about overlapping zones. Rather, I generalize your issue to be about operands that are expected to be in a particular order, but that can also be blended with blending potentially disrupting that order. Blending of operands for StemSnapH, StemSnapV or OtherBlues could lead to interpolated values that don't conform to stated ordering requirements. It looks like this is also an issue for BlueValues as well.

(These ordering requirements weren't stated in OT 1.9, but were introduced by a proposed revision of the CFF2 spec provided by Adobe that filled in missing details on hinting operators that had been specified in earlier Adobe specs, in particular ATN #5004, Adobe Type 1 Font Format](https://adobe-type-tools.github.io/font-tech-notes/pdfs/T1_SPEC.pdf). That spec didn't take variable fonts into consideration, however.)

*Blue* operators

Current draft wording for OT 1.9.1 has:

BlueValues
...

• The first integer in each pair is less than or equal to the second integer in that pair.

That wording originated from ATN #5004. For a variable font, it's unclear whether that is meant to apply only for the default values encoded in the PrivateDICT operands or also for interpolated values. It would certainly be possible for operands to be blended such that the derived first integer is greater than the derived second integer. But I'm guessing the intent was that the first value should be <= the second, even after blending.

OtherBlues
... When decoded out of deltaArray format, numbers in a pair must be in ascending order ...

(I revised the wording from the original wording in Adobe's draft, which came from ATN #5004:

... Numbers in a pair must be in ascending order...

In a deltaArray, the second operand will very often be less than the first, since it's a delta; clearly what was meant was the values to be encoded, not the deltaArray representation of those values. I guess the same would apply for the description of BlueValues.)

Again, blending could result in the derived first integer being >= the derived second integer.

I'm guessing for both of these it would be appropriate to say something along these lines,

The first value in each pair must be less than or equal to the second. (This applies to values being encoded, not their deltaArray encoded representation.) In a variable font, if either value is blended, the derived first value in each pair must still be less than or equal to the second.

StdHW, StdVW, StemSnapH, StemSnapV

I'm adding StdHW and StdVW here. While not an ordering issue, there's an issue for all of these that occurred to me while considering blending: the operands or their derived blended values could be negative. I assume they must always be non-negative, though this is not stated.

Now ordering:

• StemSnapH is an array of up to 12 numbers of the most common widths (including the dominant width given in StdHW) for horizontal stems (measured vertically). These widths must be sorted in increasing order.
• StemSnapV is an array of up to 12 numbers of the most common widths (including the dominant width given in the StdVW) for vertical stems (measured horizontally). These widths must be sorted in increasing order.

The ordering constraint originates in ATN #5004. But blending could certainly result in the derived values not being in increasing order.

Aboutt these, @skef wrote:

I believe that if you take any list of sorted numbers and linearly interpolate it with any other list of sorted numbers of the same length... the result will always be sorted.

I think that statement is true. However, blending delta values are not constrained to "other list of sorted numbers". A font development tool set could provide such a constraint, but the file format does not... unless such a requirement were explicitly added to the spec.

For BlueValues it seems clear that interpolated values in each pair are expected to be in increasing order. But in this case it's less clear whether it matters that blending for a variation instance doesn't affect the ordering. In that regard, this is similar to the issue you mention about ordering of stems.

[skef]

I think that statement is true. However, blending delta values are not constrained to "other list of sorted numbers". A font development tool set could provide such a constraint, but the file format does not... unless such a requirement were explicitly added to the spec.

I suppose this may boil down to a distinction between concrete need and academic worry. In the case of hstem(hm) and vstem(hm) you need stems to be able to move around because they can overlap and have corresponding hint masks, and therefore you need the right stem to be "active" according to the mask for certain path elements.

With StemSnap[H|V] these questions seem academic. You could have blends that change the order but there's no reason to. Probably better to just impose the narrower restriction that they must be sorted everywhere in the design space. And because alignment zones can't overlap, same with them.

[skef]

Thinking about this more, I see what you're getting at. We should add some language about the narrow restriction (ordering must be the same at all points in design space for the alignment zone operators and the StemSnap operators) while widening out the requirement on stem regions in CharStrings.

[PeterCon]

I've made revisions to address the set of issues (key changes):

1. In the description of BlueValues, revised the first bullet point of the "rules" to the following:

• The first integer in each pair must be less than or equal to the second integer in that pair. In a variable font, if either integer is blended, the first must remain less than or equal to the second for all variation instances.

2. In the description of OtherBlues, revised the paragraph after the table for the operator details to the following:

... When decoded out of deltaArray format, numbers in a pair must be in ascending order; in a variable font, blended values must remain in ascending order for all instances. Pairs must be at least 3 units apart from all other pairs, including those in the BlueValues array. (This minimum distance can be modified by the optional BlueFuzz entry in the PrivateDICT.) The restriction for the BlueValues array on the maximum difference in a pair also applies to the OtherBlues array.

3. After the table with StemSnapV details, revised the descriptions of StdHW and StdHV to the following:

• StdHW takes a number expressing the dominant width of horizontal stems (measured vertically in font design units). The operand (including a blended value in a variable font) must be non-negative.
• StdVW takes a number expressing the dominant width of vertical stems (measured horizontally in font design units). The operand (including a blended value in a variable font) must be non-negative. Typically, this will be the width of straight stems in lower case letters. (For an italic font, give the width of the vertical stem measured at an angle perpendicular to the stem direction.)

4. After that bulleted list, added the following:

Note: The ordering requirement for StemSnapH and StemSnapV applies to the values encoded as operands. In a variable font, if these operand values are blended, blending could potentially result in derived values that are not in order for some variation instances. Behavior in that situation is not defined. If a CFF2 rasterizer implementation requires stem width values in its memory representation to be sorted, it could be necessary to re-sort the stems after blended values are obtained.

5. In the Blended stem and edge hints section, replaced the note and the paragraph before it with the following:

> Note: As described above (see CharString concepts), when multiple stems are defined, the pairs of values are sorted in increasing order of the first value for each pair. This applies to the values encoded as operands of stem hint operators. In a variable font, if these operand values are blended, blending could potentially result in derived values that are not in order for some variation instances. Behavior in that situation is not defined. If a CFF2 rasterizer implementation requires stems in its memory representation to be sorted, it could be necessary to re-sort the stems after blended values are obtained, while also maintaining the association between each stem and its corresponding hintmask bit.

For the latter two, I've clarified the current ambiguity by saying explicitly that behaviour is not defined—there hasn't been discussion and consensus on whether blending must be constrained to keep pairs in order or to permit blending that results in derived pairs out of order.

See the OT 1.9.1 alpha for draft revisions addressing this issue.

[PeterConstable]

It would be nice if the spec stated an expected behavior for the cases where we are left saying, "behavior is undefined". @skef I'm guessing that current CFF2 implementations developed by Adobe (covers at least Adobe products, Windows and FreeType) probably would exhibit the same behaviours. Would it be possible to craft a test font that would make clear what their current behavior is?

[PeterConstable]

@skef: From AHG discussion last week, I gather you think the descriptions for StemSnap* and hstem* should be changed from the above to say something along these lines:

In a variable font, if these operands are blended, the derived blended stem values may [are permitted to] be in a different order. Rasterizers must maintain the association between each stem and its corresponding hintmask bit, which is based on the encoded order of default values.

Does that sound right?

[skef]

Yes, that sounds good.