READ-BDF initial changes for XCCS to MCCS #2360

MattHeffron · 2025-11-08T05:53:38Z

Verbose mode implemented incorrectly.

Cleanup DEFPACKAGE using :IMPORT-FROM, and fewer imports.
Various renaming for consistency with XCCS -> MCCS changes.
Use IL:FONTSPEC record instead of using FIRST, SECOND, etc.
Fix the parsing of IL:FONTSPEC to use COMPRESSED instead of incorrect CONDENSED.
Zero-width "image" with zero-width "escapement" GLYPHS get put into NOMAPPINGCHARSET.
Add (FILES (SYSLOAD) SYSEDIT) under existing (DECLARE: EVAL@COMPILE DONTCOPY ...)

Various renaming for consistency with XCCS -> MCCS changes. Use IL:FONTSPEC record instead of using FIRST, SECOND, etc. Fix the parsing of IL:FONTSPEC to use COMPRESSED instead of incorrect CONDENSED. Zero-width "image" with zero-width "escapement" GLYPHS get put into NOMAPPINGCHARSET. Add (FILES (SYSLOAD) SYSEDIT) under existing (DECLARE: EVAL@COMPILE DONTCOPY ...)

MattHeffron · 2025-11-08T06:07:20Z

I tried this, converting various Noto Sans, Noto Sans Mono, and Noto Serif ttf fonts to bdf and then to original-style DISPLAYFONT files.
I didn't write .MEDLEYDISPLAYFONT files. (@rmkaplan what should I change in the FONTDESCRIPTOR objects I create to write better MEDLEYFONT format files vs. pre-MEDLEYFONT FONTDESCRIPTOR.)
I'm still analyzing just how much coverage of the MCCS glyphs are covered in the Regular, Bold, Bold Italic, and Italic styles, and would they be close to full fonts.
E.g., Noto Sans Mono has only Regular and Bold, not Italic nor Bold Italic.

@hjellinek Did you determine a good set of google fonts to get good coverage of the MCCS glyphs? Anyone else

rmkaplan · 2025-11-08T07:42:51Z

On the fontdescriptor, if you've loaded all the character sets and if these are now MCCS-coded fonts, you may only just need to say that MCCS is the font charencoding. You won't be doing any further coercions, and you won't be indirecting to the charsets of other fonts/faces, so probably no need to maintain the charset source info. But I'll look more carefully.

But turns out that the prc Compare function doesn't work on your new file, the function LISPSOURCEFILEP seems to return NIL

hjellinek · 2025-11-08T20:20:12Z

I tried this, converting various Noto Sans, Noto Sans Mono, and Noto Serif ttf fonts to bdf and then to original-style DISPLAYFONT files. I didn't write .MEDLEYDISPLAYFONT files. (@rmkaplan what should I change in the FONTDESCRIPTOR objects I create to write better MEDLEYFONT format files vs. pre-MEDLEYFONT FONTDESCRIPTOR.) I'm still analyzing just how much coverage of the MCCS glyphs are covered in the Regular, Bold, Bold Italic, and Italic styles, and would they be close to full fonts. E.g., Noto Sans Mono has only Regular and Bold, not Italic nor Bold Italic.

@hjellinek Did you determine a good set of google fonts to get good coverage of the MCCS glyphs? Anyone else

Yes, take a look at the "font stacks" (that's Google/CSS terminology) that I defined in my WebFontMetrics project. The Java source file Main.java lines 56-70 might help. I'm happy to explain any of it. The companion FontCoverage tool examines font stacks for characters present in the code but missing from a font stack.

rmkaplan · 2025-11-08T20:26:13Z

One other field to set: FONTCOMPLETEP. If you have installed all possible characters in all possible character sets in the fontdescriptor before you write it out, setting this flag indicates that there are no further possible coercions ever to consider--your fonts are complete.

rmkaplan · 2025-11-08T21:17:21Z

On the prc LISPSOURCEFILEP problem, I think I was running in a system that hadn't been upgraded to the LLPACKAGE changes. It now works.

MattHeffron · 2025-11-08T23:50:45Z

I defined in my WebFontMetrics project

Herb, you claim there that Noto Sans Mono has all 4 styles, but it appears to me that it has only Regular and Bold, but neither Italic form. Am I missing something?

hjellinek · 2025-11-09T00:07:29Z

I defined in my WebFontMetrics project

Herb, you claim there that Noto Sans Mono has all 4 styles, but it appears to me that it has only Regular and Bold, but neither Italic form. Am I missing something?

I put the finishing touches on that code back in February, so I could be wrong about this explanation, but I implemented Noto Sans Mono as a FallbackFontStack, which is an object - a subclass of FontStack - that takes glyphs from the named font(s), but falls back to another font stack if any are missing. In this case I define Noto Sans Mono like this:

FontStack notoSansMono = new FallbackFontStack(notoSans, "Noto Sans Mono", "Noto Sans Mono");

meaning "Give it the descriptive name 'Noto Sans Mono', base it on the Noto Sans Mono web font, and get any missing characters from plain old Noto Sans," which I defined earlier and stuck in a local variable.

MattHeffron · 2025-11-09T20:24:09Z

@hjellinek (I have no experience with java...) I've downloaded the WebFontMetrics project from github (as the .zip, and unpacked it).
I'm trying to follow the README.md, so:

I installed java (sudo apt install openjdk-21-jre-headless),
connected to the WebFontMetrics directory,
created the destination directory (beside the WebFontMetrics directory) and
gave the java command as in the README.md (java -cp WebFontMetrics.main org.interlisp.Main -d ../wfm)

I get the error:

Error: Could not find or load main class org.interlisp.Main
Caused by: java.lang.ClassNotFoundException: org.interlisp.Main

What's wrong?

hjellinek · 2025-11-09T21:29:51Z

@hjellinek (I have no experience with java...) I've downloaded the WebFontMetrics project from github (as the .zip, and unpacked it). I'm trying to follow the README.md, so:
* I installed java (`sudo apt install openjdk-21-jre-headless`),

* connected to the WebFontMetrics directory,

* created the destination directory (beside the WebFontMetrics directory) and

* gave the java command as in the README.md (`java -cp WebFontMetrics.main org.interlisp.Main -d ../wfm`)
I get the error:
Error: Could not find or load main class org.interlisp.Main
Caused by: java.lang.ClassNotFoundException: org.interlisp.Main
What's wrong?

Hi @MattHeffron,

Java is a compiled language like C, so you need to compile (build) the application before running it.
I omitted build instructions because I assumed that was understood. But I will document it since, even if you're cognizant of the fact that you have to build the app before running it, there are multiple ways someone might try to do that.

hjellinek · 2025-11-09T22:00:21Z

Hi @MattHeffron,

Java is a compiled language like C, so you need to compile (build) the application before running it. I omitted build instructions because I assumed that was understood. But I will document it since, even if you're cognizant of the fact that you have to build the app before running it, there are multiple ways someone might try to do that.

If you pull the project now you'll find build instructions in README.md that uses a new fatJar build task that packages up all the dependencies to make life easier.

You will probably still run into trouble because you downloaded the JRE (Java Runtime Environment), which doesn't include the Java compiler, javac, as opposed to the JDK (Java Development Kit). "openjdk" is an unfortunate name for the overall project, implying that it's the full JDK. And the headless variant may not include the font machinery the program relies on. (It probably does, but why take chances?)

Instead, I suggest you remove openjdk-21-jre-headless and install openjdk-21-jdk.

MattHeffron · 2025-11-09T22:55:29Z

@hjellinek

Replaced java:

sudu apt remove openjdk-21-jre-headless
sudo apt install openjdk-21-jdk

Following README.md.
./gradlew fatJar appears to have worked.
However:

$ java -jar build/libs/WebFontMetrics-1.0-SNAPSHOT.jar org.interlisp.Main -d ../wfm
SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider]
Exception in thread "main" com.beust.jcommander.ParameterException: Was passed main parameter 'org.interlisp.Main' but no main parameter was defined in your arg class
        at com.beust.jcommander.JCommander.initMainParameterValue(JCommander.java:1028)
        at com.beust.jcommander.JCommander.parseValues(JCommander.java:819)
        at com.beust.jcommander.JCommander.parse(JCommander.java:373)
        at com.beust.jcommander.JCommander.parse(JCommander.java:352)
        at org.interlisp.Main.main(Main.java:54)

hjellinek · 2025-11-09T23:05:40Z

@hjellinek

Replaced java:

sudu apt remove openjdk-21-jre-headless
sudo apt install openjdk-21-jdk

Following README.md. ./gradlew fatJar appears to have worked. However:

$ java -jar build/libs/WebFontMetrics-1.0-SNAPSHOT.jar org.interlisp.Main -d ../wfm
SLF4J(I): Connected with provider of type [ch.qos.logback.classic.spi.LogbackServiceProvider]
Exception in thread "main" com.beust.jcommander.ParameterException: Was passed main parameter 'org.interlisp.Main' but no main parameter was defined in your arg class
        at com.beust.jcommander.JCommander.initMainParameterValue(JCommander.java:1028)
        at com.beust.jcommander.JCommander.parseValues(JCommander.java:819)
        at com.beust.jcommander.JCommander.parse(JCommander.java:373)
        at com.beust.jcommander.JCommander.parse(JCommander.java:352)
        at org.interlisp.Main.main(Main.java:54)

Oh, there was a pair of typos (fixed) in README.md. The -jar should be -cp in both command lines.
-jar means that the JAR file defines a main procedure, and to pass the command line to that. -cp means that the code is to be found in the JAR file, but I'll tell you what main procedure to run.

Because org.interlisp.Main is defined as the main procedure, you could do this:

$  java -jar build/libs/WebFontMetrics-1.0-SNAPSHOT.jar -d ../wfm

MattHeffron · 2025-11-09T23:30:18Z

@hjellinek Thanks.

$ java -jar build/libs/WebFontMetrics-1.0-SNAPSHOT.jar -d ../wfm

Seems to be working. It ran with no errors.
I noticed that in .wcm files, the widths field is :XCCS-WIDTHS. Are these XCCS or MCCS encoded?

hjellinek · 2025-11-09T23:32:23Z

They are MCCS-encoded. I'll change that keyword.

MattHeffron · 2025-11-10T21:55:49Z

@rmkaplan This is ready, essentially.
This will read BDF files and create MCCS encoded DISPLAYFONT files.

Next, (probably in a separate PR) will be constructing complete (mostly) fonts from complementary sets of BDF files.
(Which may mean figuring out how to identify which glyphs need to be taken from which .ttf file into BDF, etc., and providing the additional Medley Font Format properties in the created FONTDESCRIPTOR instances.)

@hjellinek (The above discussion should have been in an issue, not in this PR.)

rmkaplan · 2025-11-11T08:07:00Z

As displayfont files, are these single-charset AC format or Strike, or Medleyfont format? Or are the charsets already combined into a single file?

…

On Nov 10, 2025, at 1:56 PM, Matt Heffron ***@***.***> wrote: MattHeffron left a comment (Interlisp/medley#2360) <#2360 (comment)> This is ready, essentially. This will read BDF files and create MCCS encoded DISPLAYFONT files. Next, (probably in a separate PR) will be constructing complete (mostly) fonts from complementary sets of BDF files. (Which may mean figuring out how to identify which glyphs need to be taken from which .ttf file into BDF, etc., and providing the additional Medley Font Format properties in the created FONTDESCRIPTOR instances.) — Reply to this email directly, view it on GitHub <#2360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJJOMVM4XTHKLHVGSGD34ECXXAVCNFSM6AAAAACLQJDGV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMJUGA2DONBQGQ>. You are receiving this because you were mentioned.

MattHeffron · 2025-11-11T08:13:34Z

They're single-charset Strike strike format.
In this PR there are no Medley Font Format related changes except XCCS to MCCS encoding.
(And bug fixes and code cleanup.)

rmkaplan · 2025-11-11T19:48:04Z

OK, I think the next step is only to assert that those particular strike files are MCCS-encoded, since the strike files themselves can't keep that information. I could extend \READCHARSET to look at a variable MCCSFONTFAMILIES, and then you would push your family names on that list. That's the way it deals with NS fonts and Alto fonts. Or define a special READBDF.CHARSET that reads the strike file and smashes in the charencoding. After that, the code I already have will assemble all the charsets into a single medleyfont file. And then we have to put entries on the list DISPLAYFONTCHARCOERCIONS that give the back-up mappings from our existing fonts to the corresponding bdf fonts, e.g. to specify that Gacha and Terminal can be filled in from the mono font. And finally, remake Gacha.

…

On Nov 11, 2025, at 12:14 AM, Matt Heffron ***@***.***> wrote: MattHeffron left a comment (Interlisp/medley#2360) <#2360 (comment)> They're single-charset Strike strike format. In this PR there are no Medley Font Format related changes except XCCS to MCCS encoding. (And bug fixes and code cleanup.) — Reply to this email directly, view it on GitHub <#2360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJNI7I2WH57FBC6VLUL34GLEVAVCNFSM6AAAAACLQJDGV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMJVGQ3DSOBTGI>. You are receiving this because you were mentioned.

MattHeffron · 2025-11-11T20:09:43Z

@rmkaplan
My intent was first to do this change to MCCS encoding.
Next is to include information in the FONTDESCRIPTOR and CHARSETINFO objects that are present in the BDF but not included when saved in DISPLAYFONT (strike) format. E.g., the CHARSETINFO WIDTHS can be independent of CHARSETINFO IMAGEWIDTHS. (Strike format writer saves only the IMAGEWIDTHS, and the reader copies that to the WIDTHS.)
Also any other property that READ-BDF can set in the new Medley Font FONTDESCRIPTOR.
At that point it'd make sense to save the font(s) as MEDLEYDISPLAYFONT format.

I'm looking at @hjellinek 's Web Font project to see if I can use that process to (help) get "complete" fonts from ttf to BDF, and then to MEDLEYDISPLAYFONT.

MattHeffron added 6 commits August 8, 2025 22:18

Verbose mode implemented incorrectly

ea087f7

Merge branch 'master' into mth49--some-errors-parsing-BDF

e69b852

Merge branch 'master' into mth49--some-errors-parsing-BDF

61ba8b8

Merge branch 'master' into mth49--some-errors-parsing-BDF

fa81f27

Merge branch 'master' into mth49--some-errors-parsing-BDF

17292d3

MattHeffron requested a review from rmkaplan November 8, 2025 05:53

MattHeffron self-assigned this Nov 8, 2025

MattHeffron added the bug Something isn't working (as per documentation) label Nov 8, 2025

MattHeffron added this to fonts/unicode Nov 8, 2025

MattHeffron added the enhancement New feature or request label Nov 8, 2025

Merge branch 'master' into mth49--some-errors-parsing-BDF

0ffa408

Uh oh!

READ-BDF initial changes for XCCS to MCCS #2360

Are you sure you want to change the base?

READ-BDF initial changes for XCCS to MCCS #2360

Conversation

MattHeffron commented Nov 8, 2025

Uh oh!

MattHeffron commented Nov 8, 2025

Uh oh!

rmkaplan commented Nov 8, 2025

Uh oh!

hjellinek commented Nov 8, 2025

Uh oh!

rmkaplan commented Nov 8, 2025

Uh oh!

rmkaplan commented Nov 8, 2025

Uh oh!

MattHeffron commented Nov 8, 2025

Uh oh!

hjellinek commented Nov 9, 2025

Uh oh!

MattHeffron commented Nov 9, 2025

Uh oh!

hjellinek commented Nov 9, 2025

Uh oh!

hjellinek commented Nov 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MattHeffron commented Nov 9, 2025

Uh oh!

hjellinek commented Nov 9, 2025

Uh oh!

MattHeffron commented Nov 9, 2025

Uh oh!

hjellinek commented Nov 9, 2025

Uh oh!

MattHeffron commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rmkaplan commented Nov 11, 2025 via email

Uh oh!

MattHeffron commented Nov 11, 2025

Uh oh!

rmkaplan commented Nov 11, 2025 via email

Uh oh!

MattHeffron commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hjellinek commented Nov 9, 2025 •

edited

Loading

MattHeffron commented Nov 10, 2025 •

edited

Loading