Noto Nastaliq Urdu not working as embedded font in PDF

[klapauszak]

Adobe Acrobat is unable to extract or display Noto Nastaliq Urdu characters when used as an embedded PDF font.

Describe the bug
Using Noto Nastaliq Urdu within Moodle. It works perfectly as a display font for Moodle, but when attempting to embed it into a PDF (via Moodle Custom Certificate plugin) none of the Urdu characters appear. When opening the PDF, Adobe Acrobat reports "Unable to extract the embedded font 'AAAAAE+NotoNastaliqUrdu-Regular'. Some characters may not display or print correctly." Using Acrobat Preflight mode, it reports "Font reverts to .notdef glyph", "Glyph in subset font missing", and "Character references .notdef glyph".

To Reproduce
In a Moodle system, install the Custom Certificate plugin and define a certificate using Urdu characters. Use TCPDF to install Noto Nastaliq Urdu as a font for use in certificates. View the certificate, and also save it as a PDF file. Open the PDF file in Acrobat.

Expected behavior
All Urdu characters in the certificate will be blank, though characters using other fonts display correctly. When attempting to open the PDF in Acrobat, get the error messages above.

Screenshots

Additional context

[simoncozens]

Can I ask where you got the font from? Did you download it from fonts.google.com or elsewhere? Could you try with the latest build from the Noto project instead?

I've used NNU in PDF files in various pieces of software, and never had any problems with it, so my suspicions would lie with the TCPDF library. But it would be quite complicated to isolate the problem to that environment.

[klapauszak]

Hi Simon - thanks for that response. I don't recall where I got my original Noto Nataliq Urdu font, but it was probably from fonts.google.

Using the link you provided I've downloaded the latest build of NotoNastaliqUrdu-Regular from the Noto project (a couple of them, including "hinted" and "full") and tried them. There is minor progress - I no longer get the error message "Unable to extract embedded font", but it still isn't displaying Urdu characters. Although it seems that the font is now embedded in the PDF, I just get squares for each Urdu character and Adobe Preflight indicates that "Character references .notdef glyph". I've attached the PDF for your inspection.

Any clues what is happening here? Can you tell from the PDF file why the embedded font doesn't have characters for the Urdu text it's trying to display? Is it a problem with the font missing characters, or the characters being displayed?

Is there an issue with the parameters I'm using in TCPDF to load the font? I've used Font Type=Autodetect, Flags=Symbol (and I also tried Non-Symbol), and Encoding ID=Unicode - should I be using something else?

I'd appreciate any hints that you can offer - including any other Urdu fonts that might fix this issue. (In dealing with other charsets, it's been a process of trial-and-error to find a font that worked. So far I've tried 20+ Urdu fonts without success.)

Completion_Certificate - NotoNastaliqUrdu-Regular.pdf

Thanks,
Stan

[simoncozens]

Oh, if you've tried 20 Urdu fonts and none of them work, then it's definitely the PDF software! Unfortunately I'm not familiar enough with it to offer more insight, but you could try the TCPDF repo issues tracker...

[klapauszak]

Hi Simon – well, perhaps. As mentioned, I’ve had similar problems with other charsets (e.g. Burmese) but after trying 7-10 fonts I found one that worked. So font differences also contribute to this problem. This is my first try with an Arabic-based font and I know there are unique aspects to those which may add complexity. Looking at the TCPDF git site I don’t see any issues identified with Urdu, and nothing relevant related to Arabic. I suppose I can open a ticket there also, but what I really need is someone with the font expertise (which I don’t have) to peer into this PDF to understand what is the mismatch between the encoded characters and the font. Thanks, Stan From: Simon Cozens ***@***.***> Sent: August 30, 2023 12:02 PM To: google/fonts ***@***.***> Cc: klapauszak ***@***.***>; Author ***@***.***> Subject: Re: [google/fonts] Noto Nastaliq Urdu not working as embedded font in PDF (Issue #6584) Oh, if you've tried 20 Urdu fonts and none of them work, then it's definitely the PDF software! Unfortunately I'm not familiar enough with it to offer more insight, but you could try the TCPDF repo issues tracker... — Reply to this email directly, view it on GitHub <#6584 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN4JHIR3Y3R6XSRG75JSH23XX6EY3ANCNFSM6AAAAAA3AJ6F2Q> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AN4JHIQBLJK4AM3UOW2DGG3XX6EY3A5CNFSM6AAAAAA3AJ6F2SWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTFJ4ZMI.gif> Message ID: ***@***.*** ***@***.***> >

[simoncozens]

I’ve had similar problems with other charsets (e.g. Burmese) but after trying 7-10 fonts I found one that worked.

Again, if the question is "are 20 Urdu fonts and 7-10 Burmese fonts broken, or is a PHP-based PDF library broken?", I have no doubt what the answer is. I'm afraid I don't have time and resources to debug this further, but given the evidence:

• NNU works fine with every other PDF application.
• Most fonts don't work but some do.
• Numerals are working.
• It happens with Burmese as well.
• Adobe is complaining about notdefs.

my working theory would be that the PDF library expects all glyphs to be associated with Unicode codepoints.

When I was working on the SILE typesetter, I looked at a lot of PDF libraries and most of them expect the text to provided as a set of Unicode codepoints instead of a set of glyph IDs, so they were no use to me. It's just a common design flaw in text layout libraries - it works reasonably well when there is a one-to-one mapping between codepoints and glyphs (Latin, very basic Arabic), but does not work when the characters need to be decomposed into multiple (unencoded) glyphs.

Try all these symbols (which are Unicode encoded, and are not substituted to unencoded glyphs): ؏؁؎؂؃؀ ٠١٢٣٤٥٦٧٨٩۵۴۶۷

I bet they'll work.

[klapauszak]

Hi Simon Thanks, that is helpful. I tried the sample characters you provided and indeed they display correctly in the PDF (albeit with some curious re-ordering) – see attached. So perhaps what I’m dealing with is “a PDF library that requires a one-to-one correspondence between glyphs and Unicode codepoints” (and all of that is way over my head). That may be a limitation of TCPDF, which is a plugin to simplify loading of fonts into Moodle directories. But more likely it’s a limitation of the Certificate plugin, which embeds those fonts to create PDF certificates – my guess is that it’s there that the “PDF library” is created from the font. I can ask the developers of both, but all of these are supported by volunteers with limited time. In the event that I can’t fix the problem with the PDF library, is there such a thing as an Urdu font with one-to-one correspondence between glyphs and Unicode codepoints? I guess that’s what my trial-and-error method ultimately found for Burmese, but I know Arabic fonts are more complex. Thanks, Stan From: Simon Cozens ***@***.***> Sent: August 31, 2023 1:23 AM To: google/fonts ***@***.***> Cc: klapauszak ***@***.***>; Author ***@***.***> Subject: Re: [google/fonts] Noto Nastaliq Urdu not working as embedded font in PDF (Issue #6584) I’ve had similar problems with other charsets (e.g. Burmese) but after trying 7-10 fonts I found one that worked. Again, if the question is "are 20 Urdu fonts and 7-10 Burmese fonts broken, or is a PHP-based PDF library broken?", I have no doubt what the answer is. I'm afraid I don't have time and resources to debug this further, but given the evidence: * NNU works fine with every other PDF application. * Most fonts don't work but some do. * Numerals are working. * It happens with Burmese as well. * Adobe is complaining about notdefs. my working theory would be that the PDF library expects all glyphs to be associated with Unicode codepoints. When I was working on the SILE typesetter, I looked at a lot of PDF libraries and most of them expect the text to provided as a set of Unicode codepoints instead of a set of glyph IDs, so they were no use to me. It's just a common design flaw in text layout libraries - it works reasonably well when there is a one-to-one mapping between codepoints and glyphs (Latin, very basic Arabic), but does not work when the characters need to be decomposed into multiple (unencoded) glyphs. Try all these symbols (which are Unicode encoded, and are not substituted to unencoded glyphs): ؏؁؎؂؃؀ ٠١٢٣٤٥٦٧٨٩۵۴۶۷ I bet they'll work. — Reply to this email directly, view it on GitHub <#6584 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN4JHISIB73UAYM63YXE5BTXYBCU7ANCNFSM6AAAAAA3AJ6F2Q> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AN4JHIUPSJLEGVIR33UWPWTXYBCU7A5CNFSM6AAAAAA3AJ6F2SWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTFLTMCC.gif> Message ID: ***@***.*** ***@***.***> >

[simoncozens]

That may be a limitation of TCPDF, which is a plugin to simplify loading of fonts into Moodle directories. But more likely it’s a limitation of the Certificate plugin

I promise you, it's the PDF library.

is there such a thing as an Urdu font with one-to-one correspondence between glyphs and Unicode codepoints?

No, that's just not really possible as a consequence of how the Nastaliq style works. (See the section called "Shapeshifting" in this article.) You can try an ordinary Arabic font (e.g. Noto Naskh Arabic), but it won't be quite as culturally relevant for Urdu students, particularly in the context of a school certificate.

[klapauszak]

Hi Simon – and by “the PDF library”, do you mean TCPDF? Thanks, Stan From: Simon Cozens ***@***.***> Sent: August 31, 2023 2:30 PM To: google/fonts ***@***.***> Cc: klapauszak ***@***.***>; Author ***@***.***> Subject: Re: [google/fonts] Noto Nastaliq Urdu not working as embedded font in PDF (Issue #6584) That may be a limitation of TCPDF, which is a plugin to simplify loading of fonts into Moodle directories. But more likely it’s a limitation of the Certificate plugin I promise you, it's the PDF library. is there such a thing as an Urdu font with one-to-one correspondence between glyphs and Unicode codepoints? No, that's just not really possible as a consequence of how the Nastaliq style works. (See the section called "Shapeshifting" in this article <https://fonts.googleblog.com/2023/03/why-are-there-so-few-digital-nastaliq.html> .) You can try an ordinary Arabic font (e.g. Noto Naskh Arabic), but it won't be quite as culturally relevant for Urdu students, particularly in the context of a school certificate. — Reply to this email directly, view it on GitHub <#6584 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AN4JHIR2DQ6I7RDNUSRIGDDXYD67HANCNFSM6AAAAAA3AJ6F2Q> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AN4JHIU4FLVVKXFRLEJLG5LXYD67HA5CNFSM6AAAAAA3AJ6F2SWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTFN6KXC.gif> Message ID: ***@***.*** ***@***.***> >

[simoncozens]

Yes, TCPDF.